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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



Applicant: S. Okamoto et al. 
Serial No.: To Be Assigned 
Filed: Herewith 

FOR: DRIVING-OPERATION ASSIST AND 
RECORDING MEDIUM 



Art Unit: 
Examiner: 



PRELIMINARY AMENDMENT 

Assistant Commissioner for Patents 
Washington, D.C. 20231 
SIR: 

Prior to examination, please amend the above-identified application as 

follows: 

IN THE SPECIFICATION : 

After the title and before the first paragraph, please insert -THIS 
APPLICATION IS A U.S. NATIONAL PHASE APPLICATION OF PCT 
INTERNATIONAL APPLICATION PCT/JP99/05509-. 

On page 5, line 2, please delete "The 2 nd invention" and insert 
therefor —Another aspect--. 

IN THE CLAIMS : 

Please amend the claims as follows: 

1 6. (Amended) The driving-operation assist according to any one 

2 of claims 1 to [5] 4, characterized in that 

3 the assumed-movement pattern includes video data showing the 

4 relation between an assumed-movement start area which is an area in which the 

5 vehicle at start of the movement of the vehicle when performing the above 

6 predetermined series of driving operations is present and an assumed-movement 

7 end area which is an area in which the vehicle at end of the movement is present. 

1 8- (Amended) The driving-operation assist according to claim 6[ 

2 or 7], characterized in that 
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3 the assumed-movement pattern includes video data showing virtual 

4 poles arranged on the outer edge of the vehicle movement area. 

1 9. (Amended) The driving-operation assist according to [any one 

2 of] claim[s] 6 [to 8], characterized in that 

3 the synthetic-image generating means superimposes current-position 

4 data serving as video data showing an area in which the vehicle is present, on the 

5 circumferential-state image to generate the synthetic image. 

1 11. (Amended) The driving-operation assist according to [any one 

2 of] claim[s] 6 [to 10], characterized in that when actual driving operations 

3 corresponding to the above predetermined series of driving operations are started, 

4 the synthetic-image generating means thereafter fixes the positional 

5 relation between the assumed-movement pattern and the circumferential-state 

6 image at the point of time when the actual driving operations are started and 

7 generates the synthetic image. 

1 14. (Amended) The driving-operation assist according to [any one 

2 of] claimfs] 6 [to 13], characterized in that 

3 final-position inputting means for inputting a final position which is 



4 a position of the vehicle at end of the movement and start-position determining 

5 means for obtaining a start position which is a position at start of the movement 

6 corresponding to the input final position in accordance with the assumed- 

7 movement pattern are included, and 

8 the synthetic-image generating means superimposes the input final 

9 position and the start position corresponding to the input final position on the 
10 circumferential-state image to generate the synthetic image. 

1 16. (Amended) The driving-operation assist according to any one of 

2 claims 1 to [15]4, characterized in that 

3 assumed-movement-pattern storing means is included which holds 

4 data relating to the above predetermined series of driving operations and holds 

F:\CLIENT\MTS\3200US\NPhasePAmend.doc 
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5 assumed-movement data including at least the assumed-movement pattern. 

1 18. (Amended) The driving-operation assist according to claim 16 

2 [or 17], characterized in that pattern correcting means is included which is able to 

3 update and correct the content of the assumed-movement pattern held in the 

4 assumed-movement-pattern storing means. 

1 21. (Amended) The driving-operation assist according to [any one 

2 of] claim[s] 16 [to 20], characterized in that the assumed-movement data includes 

3 time-series data showing a relationship between a movement distance and a 

4 steering angle of the steering wheel of the vehicle. 

1 23. (Amended) The driving-operation assist according to [any one 

2 of] claim[s] 1 1 [to 13 or claim 22], characterized in that operation-start detecting 

3 means is included which automatically detects that actual driving operations 

4 corresponding to the above predetermined series of driving operations are started 

5 through an input from a driver or a predetermined driving operation. 

1 24. (Amended) The driving-operation assist according to [any one 

2 of] claim[s] 5 [to 23], characterized in that 

3 when the viewpoint converting means changes the predetermined 

4 viewpoint, the viewpoint converting means fixes the predetermined viewpoint to 

5 the vehicle before the actual driving operations corresponding to the 

6 predetermined series of driving operations are started and changes the 

7 predetermined viewpoint to a point fixed to the three-dimensional space after the 

8 actual driving operations corresponding to the predetermined series of driving 

9 operations are started. 

1 26. (Amended) The driving-operation assist according to any one of 

2 claims 1 to [5]4, characterized in that the assumed-movement pattern includes 

3 video data showing a circumscribed area on a space through which the vehicle 

4 passes when the predetermined series of driving operations are performed. 

1 29. (Amended) The driving-operation assist according to any one of 

F:\CLIENT\MTS\3200US\NPhasePAmend.doc 
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2 claims 1 to [5]4, characterized in that the assumed-movement patterns respectively 

3 include a change from backward movement to forward movement or from forward 

4 movement to backward movement in the predetermined series of driving 

5 operations. 

1 30. (Amended) The driving-operation assist according to any one of 

2 claims 1 to [5]4, characterized in that obstacle inputting means is included which 

3 is able to input a position of an obstacle area in an image to displaying means for 

4 displaying the synthetic image from a driver. 

1 33. (Amended) A recording medium characterized by storing a 

2 program of making a computer execute all of some of functions of each means of 

3 any one of claims 1 to [32]4. 

1 34. (Newly Added) The driving-operation assist according to claim 

2 12, characterized in that operation-start detecting means is included which 

3 automatically detects that actual driving operations corresponding to the above 

4 predetermined series of driving operations are started through an input from a 

5 driver or a predetermined driving operation. 

1 35. (Newly Added) The driving-operation assist according to 
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2 claim 13, characterized in that operation-start detecting means is included 

3 which automatically detects that actual driving operations corresponding to the 

4 above predetermined series of driving operations are started through an input 

5 from a driver or a predetermined driving operation. 

1 36. (Newly Added) The driving-operation assist according to 

2 claim 22, characterized in that operation-start detecting means is included 

3 which automatically detects that actual driving operations corresponding to the 

4 above predetennined series of driving operations are started through an input 

5 from a driver or a predetermined driving operation. 

Respectfully Submitted, 




A1 - 

Jankovitz, 
Attorneys for Appli 

JJJ/ap 
Suite 301 

One Westlakes, Berwyn 
P.O. Box 980 

Valley Forge, PA 19482-0980 
(610) 407-0700 

The Assistant Commissioner for Patents is hereby 
authorized to charge payment to Deposit Account 
No. 18-0350 of any fees associated with this 
communication. 

EXPRESS MAIL Mailing Label Number:EL492447773US 

Date of Deposit: June 6, 2000 
I hereby certify that this paper and fee are being deposited, under 37 C.F.R. § 1 . 1 0 and with sufficient postage, using 
the "Express Mail Post Office to Addressee" service of the United States Postal Service on the date indicated above 
and that the deposit is addressed to the Assistant Commissioner for Patents, Washington, L>Q. 2023 1 . 



Kathleen Libby 
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DESCRIPTION 



DRIVING— OPERATION ASSIST AND RECORDING MEDIUM 



Technical Field 

The present invention relates to a driving-operation 
assist for assisting driving operations of a vehicle and 
a recording medium for storing a program for making a 
computer execute all or some of functions of each means 
of the driving-operation assist. 



Background Art 

A conventional general driving-operation assist 
estimates a moving trace of a vehicle corresponding to 
a steering angle of a steering wheel for backward movement 
of the vehicle by a steering sensor for detecting the 
steering angle of the steering wheel . When a vehicle moves 
backward, an image of a rear or side-rear field of view 
photographed by a camera is displayed and moreover, when 
the steering wheel is operated, a moving trace of the 
vehicle estimated correspondingly to the steering angle 
of the steering wheel is superimposed on the image of the 
rear or side-rear field of view. According to the assist, 
driving operations by a driver will be performed as 
described below. That is, a driver moves a vehicle to 
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a place where the vehicle can be probably parked while 
fixing the steering wheel of the vehicle. Then, at the 
place, the driver finds a steering angle capable of moving 
the vehicle to a space for parking the vehicle without 
operating any steering wheel while confirming a 
vehicle-moving trace estimated by operating the steering 
wheel. Then, by moving the vehicle backward toward a 
parking space while keeping the steering angle, parking 
is theoretically completed. 

A conventional example of the above driving-operation 
assist is disclosed in the official gazette of Japanese 
Patent Laid-Open No. 1-14700. 

However, to park a vehicle by using the above assist, 
it is necessary to find a place where the vehicle can be 
moved to a parking space and then determine a steering 
angle at which the steering wheel is fixed. To master 
these operations, skill is necessary. Moreover, when the 
size of a vehicle to be driven is changed, a sense differs. 
Therefore, the driving know-how accumulated during skill 
is not greatly useful. 

Incidentally, to park a vehicle, it is generally 
difficult to complete parking operations while keeping 
a steering angle of a steering wheel constant from start 
of the parking operations except a case in which there 
is no obstacle nearby. For example, to perform parallel 



parking, a driver first moves a vehicle backward by turning 
the steering wheel in a proper direction while moving the 
vehicle to a place for parking the vehicle from a 
parking-operation start position and then, moves the 
vehicle to a target place by inversely turning the steering 
wheel when the vehicle properly moves backward. That is, 
in case of taking parallel parking as an example, it is 
difficult to park a vehicle while keeping a steering angle 
of a steering wheel constant. 

Moreover, as for a conventional assist, if a driver 
slightly moves the steering wheel of a vehicle, a moving 
route of the vehicle re-estimated by the slight 
steering-angle change is displayed. Therefore, the 
driver may be confused. 

That is, as for a conventional driving-operation 
assist, a driver cannot intuitively find a place where 
a vehicle can be easily moved to a parking space at a glance 
while confirming an image obtained by synthesizing the 
parking space, a state around the space, and a route for 
guiding the vehicle to the parking space. 

Disclosure of the Invention 

The present invention is made to solve the above 
problems of an conventional driving-operation assist and 
its object is to provide a driving-operation assist by 



which a driver can directly confirm the relation between 
movements of a vehicle according to a predetermined series 
of driving operations and the circumferential state by 
displaying movements of the vehicle when the driver 
performs the predetermined series of driving operations 
together with the circumferential state and the load of 
the driver can be reduced. 

Furthermore, it is another object of the present 
invention to provide a recording medium for storing a 
program for making a computer execute all or some of 
functions of each means of the above driving-operation 
assist. 

To solve the above described problems the present 
invention provides a driving-operation assist 
characterized by comprising: 

circumferential-state imaging means for imaging a 
circumferential state of a vehicle with a camera and 
generating a circumferential-state image and/or storing 
the generated circumferential-state image; 

synthetic-image generating means for generating a 
synthetic image by superimposing on the 

circumferential-state image , an assumed-movement pattern 
which is the video data showing movement of the vehicle 
in case of performing a predetermined series of driving 
operations for the vehicle; and 
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displaying means for displaying the synthetic image. 

The 2 nd invention of the present 
invention (corresponding to claim 26[sic; claim 33]) is 
a recording medium characterized by storing a program for 
making a computer execute all or some of functions of each 
means of the above driving-operation assist. 

Brief Description of the Drawings 

FIG. 1 is a block diagram showing a configuration of 
a driving-operation assist of a first embodiment of the 
present invention; 

FIG. 2 shows a top view and an elevation view of a 
vehicle provided with cameras of an imaging section 101 
of a driving-operation assist of the first embodiment of 
the present invention; 

FIG. 3 is an elevation view showing a viewpoint of 
a virtual camera of the driving-operation assist of the 
first embodiment of the present invention; 

FIG. 4 is an illustration showing a 
circumferential-state image viewed from the virtual camera 
of the driving-operation assist of the first embodiment 
of the present invention; 

FIGS. 5(a) and 5(b) are illustrations showing 
assumed-movement data of the driving-operation assist of 
the first embodiment of the present invention; 
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FIGS. 6(a) and 6(b) are illustrations showing 
assumed-movement data of the driving-operation assist of 
the first embodiment of the present invention; 

FIG. 7 is an illustration showing a synthetic image 
of the driving-operation assist of the first embodiment 
of the present invention; 

FIG. 8 is an illustration showing a synthetic image 
of the driving-operation assist of the first embodiment 
of the present invention; 

FIG. 9 is an illustration showing movement of a vehicle 
at the time of performing parallel parking to the left 
side; 

FIGS. 10(a) to 10(c) are illustrations showing 
synthetic images of the driving-operation assist of the 
first embodiment of the present invention; 

FIG. 11 shows illustrations of variation of 
assumed-movement patterns stored in 
assumed-movement-pattern storing means 108 of the 
driving-operation assist of the first embodiment of the 
present invention ; 

FIG. 12 is an illustration showing a modification of 
the synthetic image of the driving-operation assist of 
the first embodiment of the present invention; 



FIG. 13 is a block diagram showing a configuration 
of a driving-operation assist of a second embodiment of 
the present invention; 

FIGS. 14(a) to 14(c)- are illustrations showing 
synthetic images of the driving-operation assist of the 
second embodiment of the present invention; 

FIG. 15 is a block diagram showing a configuration 
of a driving-operation assist of a third embodiment of 
the present invention; 

FIGS. 16(a) to 16(c) are illustrations showing 
synthetic images of the driving-operation assist of the 
third embodiment of the present invention; 

FIG. 17 is a block diagram showing a configuration 
of a driving-operation assist of a fourth embodiment of 
the present invention; 

FIG. 18 is a block diagram showing a configuration 
of a driving-operation assist of a fifth embodiment of 
the present invention; 

FIGS. 19(a) to 19(c) are illustrations showing 
synthetic images of the driving-operation assist of the 
fifth embodiment of the present invention; 

FIG. 20 is a block diagram showing a configuration 
of a driving-operation assist of a sixth embodiment of 
the present invention; 
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FIGS. 21(a) to 21(c) are illustrations showing 
synthetic images of the driving-operation assist of the 
sixth embodiment of the present invention; 

FIG. 22 is a block diagram of a configuration of a 
modification of the driving-operation assist of the sixth 
embodiment of the present invention; 

FIG. 2 3 is a block diagram showing a configuration 
of a driving-operation assist of a seventh embodiment of 
the present invention; 

FIG. 2 4 is a conceptual view showing a mapping table 
stored in a mapping table 2302 of the driving-operation 
assist of the seventh embodiment of the present invention; 

FIG. 25 is a block diagram showing a configuration 
of a driving-operation assist of an eighth embodiment of 
the present invention; 

FIGS. 26(a) to 26(c) are illustrations showing 
synthetic images of the driving-operation assist of the 
eighth embodiment of the present invention; 

FIG. 27 is a block diagram showing a configuration 
of a driving-operation assist of a ninth embodiment of 
the present invention; 

FIGS. 28(a) to 28(c) are illustrations showing 
synthetic images of the driving-operation assist of the 
ninth embodiment of the present invention; 
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FIG. 29 is a block diagram showing a configuration 
of a driving-operation assist of a tenth embodiment of 
the present invention; 

FIGS. 30(a) and 30(b) are illustrations showing 
synthetic images of the driving-operation assist of the 
tenth embodiment of the present invention; 

FIG. 31 is a block diagram showing a configuration 
of a driving-operation assist of an eleventh embodiment 
of the present invention; 

FIGS. 32(a) and 32(b) are illustrations showing 
synthetic images of the driving-operation assist of the 
eleventh embodiment of the present invention; 

FIG. 33 is an illustration showing a synthetic image 
of the driving-operation assist of the eleventh embodiment 
of the present invention; 

FIG. 34 is an illustration showing a synthetic image 
of the driving-operation assist of the eleventh embodiment 
of the present invention; 

FIG. 35 is an illustration showing a synthetic image 
of the driving-operation assist of the eleventh embodiment 
of the present invention; 

FIG. 36 is a graph for explaining a contact-hazard 
evaluation function of the driving-operation assist of 
the eleventh embodiment of the present invention; 
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FIG. 37 is an illustration showing a synthetic image 
of the driving-operation assist of the eleventh embodiment 
of the present invention; 

FIG. 38 is an illustration [sic; expression] showing 
a contact-hazard evaluation function of the 
driving-operation assist of the eleventh embodiment of 
the present invention; 

FIG. 39 shows examples of assumed movement data of 
the driving-operation assist of the eleventh embodiment 
of the present invention; 

FIG. 40 is an illustration showing a synthetic image 
of the driving-operation assist of the eleventh embodiment 
of the present invention; 

FIG. 41 is an illustration showing a synthetic image 
of the driving-operation assist of the eleventh embodiment 
of the present invention; 

FIG. 42 is a block diagram showing a configuration 
of a driving-operation assist of a twelfth embodiment of 
the present invention; 

FIG. 43 is an illustration showing a synthetic image 
of the driving-operation assist of the twelfth embodiment 
of the present invention; 

FIG. 44 is a block diagram showing a configuration 
of the driving-operation assist of the twelfth embodiment 
of the present invention; 
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FIG. 45 is an illustration showing a synthetic image 
of the driving-operation assist of the twelfth embodiment 
of the present invention; and 

FIG. 46 is an illustration showing a synthetic image 
of the driving-operation assist of the twelfth embodiment 
of the present invention. 

[Description of Symbols] 

101 Imaging section 

102 Superimposing means 

103 Camera parameter table 

104 Space reconfiguring means 

105 Space-data buffer 

106 Viewpoint converting means 

107 Displaying means 

108 Assumed-movement-pattern storing means 

1301 , 1501 Start detecting means 

1302 , 1703 Integrating means 

1303, 1503, 1704 Space converting means 
1502 Image tracking means 

1701 Start inputting means 

1702 Driving controlling means 
1801 Trace correcting means 
2001 CG-image synthesizing means 
2201 Camera 



- 12 - 

2301 Mapping means 

2302 Mapping table 

2501 Final-position inputting means 

2502 Start-position determining means 

2503 Space fixing means 

Best Mode for Carrying Out the Invention 

Embodiments of the present invention will be described 
below by referring to the accompanying drawing. 

(First embodiment) 

First, a first embodiment of the present invention 
will be described by referring to the drawings . 

FIG. 1 is a block diagram showing a configuration of 
the driving-operation assist of this embodiment. The 
driving-operation assist of this embodiment is mainly used 
to assist driving operations for garaging or parallel 
parking and the like. 

As shown in FIG. 1, the driving-operation assist of 
this embodiment is configured by an imaging section 101 
comprising N cameras (camera 1 to camera N) , a camera 
parameter table 103 for storing camera parameters which 
are characteristics of the cameras, space configuring 
means 104 for generating space data relating each pixel 
configuring an image output from each camera to a point 
in a three-dimensional space in accordance with a camera 
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parameter, viewpoint converting means 106 for generating 
an image viewed from a predetermined viewpoint as a 
circumferential-state image by referring to the space data, 
a space-data buffer 105 for temporarily storing the space 
data, assumed-movement-pattern storing means 108 for 
storing assumed-movement data including assumed-movement 
patterns, superimposing means 102 for superimposing the 
assumed-movement pattern on the circumferential-state 
image and generating a synthetic image, and displaying 
means 107 for displaying a synthetic image. 

An assembly including the imaging section 101, camera 
parameter table 103, space reconfiguring means 104, and 
viewpoint converting means 106 corresponds to 
circumferential-state imaging means of the present 
invention and the superimposing means 102 corresponds to 
synthetic-image generating means of the present invention . 

First , a detailed configuration of the imaging section 
101 and a procedure in which a circumferential-state image 
of the present invention is generated in accordance with 
video data imaged by the imaging section 101 will be 
described below. 

FIG. 2 shows a top view and an elevation view of a 
vehicle provided with cameras of the imaging section 101. 
In this example, six cameras 201 to 2 06 are arranged on 
the roof portion of a vehicle as N=6. The six cameras 



- 14 - 

201 to 206 are arranged so that apart of each camera overlaps 
with a part of the imaging range of other camera and no 
dead angle is produced flatly. 

The camera parameter table 103 stores camera 
parameters of each of the above cameras (parameters showing 
camera characteristics such as setting position, setting 
angle, lens-strain correction value, and focal distance 
of camera) . The space reconfiguring means 104 generates 
space data relating each pixel configuring an image output 
from each camera in accordance with the camera parameters 
to a point in a three-dimensional space on the basis of 
a vehicle. The space-data buffer 105 temporarily stores 
the above space data and the viewpoint converting means 
106 generates an image viewed from an optional viewpoint 
such as a viewpoint of a virtual camera 301 shown in FIG. 
3 as a circumferential-state image by referring to space 
data and thereby synthesizing pixels. 

FIG. 4 shows a circumferential-state image viewed from 
the viewpoint of the virtual camera 301 shown in FIG. 3. 
This example shows a case of performing parallel parking, 
in which two vehicles currently parked are shown on the 
circumferential-state image as an obstacle 401 and an 
obstacle 402. 

Then, a procedure will be described in which the 
superimposing means 102 generates a synthetic image of 
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the present invention and finally, the displaying means 
107 displays the synthetic image. 

The assumed-movement-pattern storing means 108 
stores an assumed-movement pattern which is video data 
showing movement of a vehicle to which a typical vehicle 
driving operation is applied and time-series data showing 
the relation between vehicle moving distance (moving 
distance due to rotation of tire) and steering angle of 
steering wheel (rotation angle of steering wheel) as 
assumed-movement data of the present invention. 

Among stored assumed-movement data, the 
assumed-movement data for performing the driving operation 
of parallel parking to the left side is shown in FIG. 5 
and the assumed-movement data for performing the driving 
operation for garaging to the right side is shown in FIG. 
6. FIGS. 5(a) and 6(a) respectively show an 
assumed-movement pattern which is video data corresponding 
to a driving operation according to the time-series data 
shown in FIGS. 5(b) and 6(b) and showing operation start 
positions 501 and 601 (corresponding to an 
assumed-movement start area of the present invention), 
operation end positions 502 and 602 (corresponding to an 
assumed-movement end area of the present invention), and 
tire traces 503 and 603 (corresponding to video data showing 
vehicle tire traces of the present invention) . 
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First, a driver selects one of the assumed-movement 
patterns stored in the assumed-movement-pattern storing 
means 108 by pattern selecting means (not illustrated) . 
The superimposing means 102 superimposes a selected 
assumed-movement pattern (for example, FIG. 5(a)) on a 
circumferential-state image generated by the viewpoint 
converting means 106 (for example, FIG. 4) to synthesize 
them and generate a synthetic image of the present invention , 
and the displaying means 107 displays the synthetic image. 
In this case, for example, by adjusting the operation start 
position 501 in FIG. 5(a) to the current position of the 
vehicle, the operation end position 502 becomes an 
operation end position when starting a driving operation 
corresponding to the assumed-movement pattern from the 
current position, that is, a parking position. 

FIG . 7 shows a synthetic image obtainedby synthesizing 
the assumed-movement patterns shown in FIG. 5 and FIG. 
8 shows a synthetic image obtained by synthesizing the 
assumed-movement patterns shown in FIG. 6. 

That is, in FIG. 7 (FIG. 8), the driver can perform 
parallel parking to the left side (garaging to the right 
side) by moving the vehicle to a start position 701 (801) 
where the obstacles 401 and 402 (803 and 804) do not 
interfere with a parking position 702 ( 802 ) , the tire trace 
503 (603), and the start position 701 (801) and starting 
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a series of driving operations according to time-series 
data from the position so as to park the vehicle at the 
parking position 702 (802). 

A detailed procedure for moving the vehicle to the 
start position 701 (801) will be described below by using 
a case of performing parallel parking to the left side 
as an example and referring to FIG. 9 and FIGS. 10(a) to 
10(c). 

FIG . 9 is an illustration showing movement of a vehicle 
when performing parallel parking to the left side. As 
shown in FIG. 9, to park the vehicle at a target parking 
position 902 , a driver of the vehicle must move the vehicle 
located at a current position 901 to a target start position 
903 by assuming the operation start position 501 when 
adjusting the operation end position 502 of the 
assumed-movement pattern (FIG. 5(a)) for performing 
parallel parking to the left side to the target parking 
position 902 as the target start position 903. 

The relative positional relation between the 
operation end position 502 and the operation start position 
501 in FIG. 5(a) corresponds to a case of performing a 
driving operation in accordance with the time-series data 
in FIG. 5(b). As for an actual driving operation, fine 
adjustment can be made in accordance with fine adjustment 
of a currently-operated steering wheel. 
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When the driving-operation assist of this embodiment 
is not used, the driver must move the vehicle to the target 
start position 903 by assuming the obstacles 401 and 402 
and the target parking position 902 in accordance with 
a scene which can be confirmed from the inside of the vehicle 
through direct viewing or a mirror. In this case, there 
is a problem that an operation for the driver to assume 
the obstacles 401 and 402 and the target parking position 
902 in accordance with the scene which can be confirmed 
from the inside of the vehicle through direct viewing or 
a mirror requires skill. Moreover, even when a size of 
a vehicle or a position of a mirror is changed, there is 
a problem that a driver cannot easily correspond to the 
change . 

However, when the driving-operation assist of this 
embodiment is used, the assumed-movement pattern shown 
in FIG. 5(a) is superimposed on the circumferential-state 
image viewed from a viewpoint of a virtual camera as if 
being imaged from a position just above the vehicle as 
shown in FIG. 4 and thereby, the synthetic image shown 
in FIG. 7 is generated and displayed for a driver. 

Therefore, at the time of performing a driving 
operation for moving the vehicle to the target start 
position 903 in FIG. 9, the operation end position 502 
when assuming the current position 901 as the operation 
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start position 501 is displayed as a parking position 1001 
corresponding to the current position together with tire 
traces by displaying adjusting the current position 901 
of the vehicle to the operation start position 501 in FIG. 
5(a) as shown in FIGS. 10(a) to 10(c). When the vehicle 
is located at the current position 901 where the parking 
position 1001 coincides with the target parking position 
9 02 , movement to the target start position 903 is completed. 

That is, if a parking operation is started from the 
position when the synthetic image 1 in FIG. 10(a) is 
displayed on the displaying means 107, the parking position 
1001 overlaps with the obstacle 402. Therefore, a driver 
can confirm at a glance that the parking operation must 
be started from a position where the vehicle is further 
moved forward (upward in FIG. 10(a)). 

Moreover, if a parking operation is started from the 
position when the synthetic image 2 in FIG. 10(b) is 
displayed on the displaying means 107, a tire trace overlaps 
with the obstacle 401. Therefore, the driver can confirm 
at a glance that the parking operation must be started 
from a position where the vehicle is further moved backward 
(downward in FIG. 10(a)). 

Furthermore, if a parking operation is started from 
the position when the synthetic image 3 in FIG. 10(c) is 
displayed on the displaying means 107, a tire trace does 
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not overlap with the obstacle 401 or 402 and the driver 
can confirm at a glance that the parking position 1001 
is a position suitable for parking. Therefore, it is 
possible to confirm that a parking operation can be started 
from the position. 

Thus, by virtually synthesizing images showing the 
positional relation between a vehicle viewed from a 
position just above the vehicle, circumferential obstacles, 
parking end position, and tire traces and showing a 
synthesized image to a driver, the driver can directly 
confirm the positional relation between them at a glance. 
As a result, the driver can confirm a place suitable for 
start of a parking operation at a glance, easily move a 
vehicle to the place, and start a parking operation. 
Therefore, it is possible to park the vehicle at a purposed 
position more safely and accurately. 

Assumed-movement-pattern-operation start and end 
positions and a tire trace are intrinsic to each vehicle. 
For example, they are greatly different in small vehicle 
and large vehicle. This can be settled by storing an 
assumed-movement pattern in the assumed-movement-pattern 
storing means 108 in FIG. 1 every vehicle. Therefore, 
a driver can perform driving operations while confirming 
the relation between an assumed-movement pattern 
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corresponding to the vehicle and a circumferential 
obstacle even if the vehicle is changed to another one. 

Moreover, when vehicles are changed, it is considered 
that positions of and the number of vehicle-mounted cameras 
shown in FIG. 2 are also changed. However, this can also 
be settled by the fact that the camera parameter table 
103 in FIG. 1 stores camera parameters of each camera every 
vehicle and an image shown to a driver is not directly 
influenced. Therefore, the driver can perform driving 
operations while confirming the relation between an 
assumed-movement pattern corresponding to a vehicle 
displayed in a synthetic image almost similarly displayed 
before vehicles are changed and a circumferential obstacle 
even if vehicles are changed and thereby, camera positions 
are changed. 

As described above, according to this embodiment, it 
is possible to directly confirm a driving vehicle, an 
obstacle, and a purposed position at a glance in a driving 
operation such as parallel parking conventionally 
requiring a considerable skill for a driver, perform more 
safe and accurate driving operation, and greatly decrease 
an operation load of the driver. 

Moreover, a driver can perform a driving operation 
while confirming the relation between an assumed-movement 
pattern corresponding to a vehicle and a circumferential 
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obstacle or the like, almost similarly to a state before 
vehicles are changed even if vehicles are changed and 
thereby, it is possible to greatly decrease a load of a 
driver's skill for change of vehicles. 

FIG . 1 1 shows a variation of as sumed-movement patterns 
stored in the assumed-movement-pattern storing means 108 
in FIG. 1. The patterns are right and left 
parallel-parking assumed-movement patterns 1101 and 1102 
and right and left garaging assumed-movement patterns 1103 
and 1104. As described above, a driver selects any one 
of these patterns by pattern selecting means (not 
illustrated) . Areas to be displayed as synthetic images 
are also determined as shown by outer frames of the 
assumed-movement patterns 1101 to 1104 in FIG. 11 
correspondingly to the above four assumed-movement 
patterns . That is , an operation start position is assumed 
as a current vehicle position and a rectangular area 
including a tire trace and an operation end position is 
assumed as a synthetic-image area. 

Though a vehicle is not imaged from a vehicle-mounted 
camera in general, it is permitted in this case to hold 
CG data for a vehicle and actual-vehicle data and display 
these data in a synthetic image by superimposing them on 
each other similarly to trace data. 
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For this embodiment, assumed-movement patterns of the 
present invention are described as video data showing an 
operation start position ( assumed-movement start area of 
the present invention) , an operation end position 
(assumed-movement end area of the present invention) , and 
tire traces (image data showing vehicle tire traces of 
the present invention). However, other pattern is also 
permitted, for example, it can be that traces (video data 
showing a vehicle moving area of the present invention) 
in which a projection of a vehicle moves are included 
instead of or together with tire traces. In short, it 
is only necessary that assumed-movement patterns of the 
present invention serve as video data showing movement 
of a vehicle when assuming that a predetermined series 
of driving operations are previously applied to the 
vehicle. 

Moreover, as described above, if directly displaying 
a tire trace and/or a vehicle moving area, there is no 
margin for contact between the vehicle and an obstacle 
or the like. Therefore, as shown in FIG. 12, it is also 
permitted to display a margin line 1201 set outward by 
a predetermined value (e.g. 50 cm) from a tire trace or 
the outer margin of a vehicle moving area. 

Furthermore , for this embodiment , it is described that 
a circumferential-state image of the present invention 
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is obtained by synt he s i z ing video data imaged by the imaging 
section 101 in real time. However, at a place where the 
same driving operation is frequently performed and 
circumferential state is hardly fluctuated, other pattern 
is also permitted, for example, it can be to store the 
data produced in accordance with already-imaged video data 
in the space-data buffer 105 and use the data. 
(Second embodiment) 

Then, a second embodiment of the present invention 
will be described below by referring to the accompanying 
drawings . 

FIG. 13 is a block diagram showing a configuration 
of the driving-operation assist of the second embodiment 
of the present invention. The driving-operation assist 
of this embodiment is also mainly used to assist driving 
operations such as garaging and parallel parking. 
Therefore, in this embodiment, it is assumed that a 
component with no explanation is the same as the case of 
the first embodiment and a component provided with the 
same symbol as the case of the first embodiment has the 
function as the case of the first embodiment unless 
otherwise explained. Moreover, it is assumed that each 
modified example described for the first embodiment can 
be applied to this embodiment by applying the same 
modification unless otherwise described. 
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As shown in FIG. 13, the configuration of the 
driving-operation assist of this embodiment is different 
from that of the driving-operation assist of the first 
embodiment in that start detecting means 1301, integrating 
means 1302, and space converting means 1303 are included. 

The start detecting means 1301 to which a gear signal 
showing forward movement or backward movement and a 
steering-wheel-turning-angle signal showing a 
front-wheel steering angle are input judges that a driving 
operation (parking operation) corresponding to an 
assumed-movement pattern stored in the 

assumed-movement-pattern storing means 108 is started when 
a gear signal shows a backward-movement state and a 
front-wheel steering angle according to a 
steering-wheel-turning-angle signal reaches a certain 
value or more, which corresponds to operation start 
detecting means of the present invention. 

The integrating means 1302 integrates the 
steering-wheel rotation angle and the number of 
revolutions of rear wheels and computes a spatial movement 
change of a vehicle up to the current point of time after 
a driving operation (parking operation) is started, which 
corresponds to moving-position computing means of the 
present invention. 
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The space converting means 1303 moves an 
assumed-movement pattern in accordance with the spatial 
movement change and an assembly of the superimposing means 
102 and space converting, means 1303 corresponds to 
synthetic-image generating means of the present invention . 

In this embodiment, a procedure until a 
circumferential-state image of the present invention is 
generated in accordance with video data imaged by the 
imaging section 101 is the same as the procedure described 
for the first embodiment. Moreover, a procedure until 
an actual driving operation corresponding to an 
assumed-movement pattern is started in a procedure until 
the superimposing means 102 generates a synthetic image 
of the present invention and the displaying means 107 
displays the synthetic image is the same as that described 
for the first embodiment. 

A procedure until the superimposing means 102 
generates a synthetic image of the present invention and 
the displaying means 107 displays the synthetic image after 
an actual driving operation corresponding to an 
assumed-movement pattern is started is described below 
by using a case of performing parallel parking to the left 
side as an example. 

When a driver starts a driving operation corresponding 
to parallel parking to the left side, a gear signal shows 
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a backward-movement state and a front -wheel steering angle 
according to a steering-wheel-turning-angle signal 
reaches a certain value or more. Therefore, the start 
detecting means 1301 judges that a driving operation 
(parking operation) corresponding to parallel parking to 
the left side is started, communicates to the integrating 
means 1302 that the driving operation (parking operation) 
is started, and thereafter inputs the 
steering-wheel-rotation-angle signal and the 
number-of-revolution-of-rear-wheel signal to the 
integrating means 1302. 

The integrating means 1302 integrates the input 
steering-wheel-rotation-angle signal and 
number-of-revolution-of -rear-wheel signal after start of 
the driving operation and computes the positional relation 
between a current vehicle position 1402 and a 
parking-operation start position 1401 as shown in FIG. 
14(a). 

In accordance with the computed positional relation, 
the space converting means 1303 moves an assumed-movement 
pattern 1403 corresponding to parallel parking to the left 
side so that the operation start position (501 in FIG. 
5) coincides with the parking-operation start position 
1401 as shown in FIG. 14(b). In other words, the space 
converting means 1303 spatially fixes the assumed-movement 
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pattern 1403 at a position when the parking operation starts 
after start of the driving operation. 

After start of a driving operation, the superimposing 
means 102 synthesizes the assumed-movement pattern 1403 
spatially fixed at the parking-operation start position 
and the current vehicle position 1402 by superimposing 
the pattern 1403 and the position 1402 on a 
circumferential-state image to generate a synthetic image 
of the present invention as shown in FIG. 14(c). The 
displaying means 107 displays the synthetic image. 
Because circumferential-state images such as the obstacles 
401 and 402 are naturally fixed in a space, the positional 
relation between the circumferential-state images and the 
assumed-movement pattern 1403 is fixed in the synthetic 
image. Moreover, because a synthetic image is an image 
viewed from a viewpoint fixed on to a vehicle, only the 
current vehicle position 1402 is fixed on the screen when 
the vehicle moves and circumferential-state images and 
the assumed-movement pattern 1403 are displayed while they 
relatively move in FIG. 14(c). 

That is, according to this embodiment, a 
circumferential-state image viewed from a viewpoint of 
a virtual camera moves in accordance with movement of an 
actual vehicle and the superimposed and synthesized 
assumed-movement pattern 1403 also moves in accordance 
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with movement of a vehicle computed by the integrating 
means 1302. Therefore, the circumferential-state image 
and the assumed-movement pattern 1403 show coincident 
movement . Because a driver only has to operate a steering 
wheel along a tire trace of an assumed -movement pattern 
displayed at each point of time, a simpler and safer vehicle 
operation is realized. 
(Third embodiment) 

Then, a third embodiment of the present invention will 
be described below by referring to the accompanying 
drawings . 

FIG. 15 is a block diagram showing a configuration 
of the driving-operation assist of the third embodiment 
of the present invention. The driving-operation assist 
of this embodiment is also mainly used to assist driving 
operations for garaging, parallel parking and the like. 
Therefore, in this embodiment, it is assumed that a 
component with no explanation is the same as the case of 
the first embodiment and a component provided with the 
same symbol as the case of the first embodiment has the 
same function as the case of the first embodiment unless 
otherwise explained. Moreover, each modified example 
described for the first embodiment can be applied to this 
embodiment by applying the same modification unless 
otherwise described. 
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As shown in FIG. 15, the configuration of the 
driving-operation assist of this embodiment is different 
from that of the driving-operation assist of the first 
embodiment in that start .detecting means 1501 and image 
tracking means 1502, and space converting means 1503 are 
included. 

The start detecting means 1501 has the same functions 
as the start detecting means 1301 in FIG. 13 described 
for the second embodiment except that a 
steering-wheel-turning-angle signal or a 
number-of-revolution-of-rear-wheel signal is not output 
to any other unit. 

The image tracking means 1502 stores positional 
information of the whole or a part (e.g. operation end 
position) of video data for assumed-movement patterns on 
the basis of the whole or a part (e.g. obstacle) of video 
data for circumferential-state images on a synthetic image 
when a driving operation (parking operation) is started 
and corresponds to positional-information storing means 
of the present invention. 

The space converting means 1503 moves an 
assumed-movement pattern in accordance with the positional 
information and an assembly configured by the 
superimposing means 102 and space converting means 1503 
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corresponds to synthetic-image generating means of the 
present invention. 

In this embodiment, a procedure until a 
circumferential-state image of the present invention is 
generated in accordance with video data imaged by the 
imaging section 101 is the same as that described for the 
first embodiment. Moreover, a procedure until an actual 
driving operation corresponding to an assumed-movement 
pattern is started in a procedure until the superimposing 
means 102 generates a synthetic image of the present 
invention and the displaying means 107 displays the 
synthetic image is the same as that described for the first 
embodiment . 

A procedure until the superimposing means 102 
generates a synthetic image of the present invention and 
the displaying means 107 displays the synthetic image after 
an actual driving operation corresponding to an 
assumed-movement pattern is started is described below 
by using a case of performing parallel parking to the left 
side as an example. 

When a driver starts a driving operation corresponding 
to parallel parking to the left side, a gear signal shows 
a backward-movement state and a front-wheel steering angle 
according to a steering-wheel-turning-angle signal 
reaches a certain value or more. Therefore, the start 
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detecting means 1501 judges that a driving operation 
(parking operation) corresponding to parallel parking to 
the left side is started and communicates to the image 
tracking means 1502 that.the driving operation (parking 
operation) is started. 

When the image tracking means 1502 receives the 
information showing that the driving operation (parking 
operation) is started, there is obtained the video data 
for an end-position circumferential image 1603 including 
a part of the obstacle 402 which is a part of the video 
data for a circumferential-state image and a 
parking-operation end position 1602 on the then synthetic 
image (FIG. 16(a)) through the space-data buff er 105, and 
the video data is stored. After start of the driving 
operation, the means 1502 finds the portion concerned of 
the obstacle 402 from a circumferential-state image at 
each point of time (obtained through the space-data buffer 
105), adjusts the portion concerned of the obstacle 402 
in the end-position circumferential image 1603 to the 
portion concerned of the obstacle 402 in a 
circumferential-state image, and thereby determines the 
positional relation between the parking-operation end 
position 1602 and the circumferential-state image at that 
point of time. That is, the image tracking means 1502 
tracks the positional relation between the 
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parking-operation end position 1602 and the 
circumferential-state image at each point of time. 

According to the positional relation, the space 
converting means 1503 moves an assumed-movement pattern 
corresponding to parallel parking to the left side so that 
the operation end position (502 in FIG. 5) coincides with 
the parking-operation end position 1602 . In other words, 
after start of the driving operation, the space converting 
means 1503 spatially fixes the assumed-movement pattern 
to a position at start of the parking operation. 

As shown in FIG. 16(c), after start of the driving 
operation, the superimposing means 102 superimposes the 
assumed-movement pattern 1605 spatially-fixed to the 
position at start of the parking operation and the current 
vehicle position 1604 on a circumferential-state image 
and synthesizes them to generate a synthetic image of the 
present invention. The displaying means 107 displays the 
synthetic image. Because circumferential-state images 
of the obstacles 401 and 402 or the like, are naturally 
fixed in a space, the positional relation between the 
circumferential-state image and the assumed-movement 
pattern 1403 is fixed in the synthetic image. 

Moreover, in FIG. 16(c), because the synthetic image 
is an image viewed from a viewpoint fixed on a vehicle, 
if the vehicle moves, only the current vehicle position 
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1604 is fixed on the screen and the circumferential-state 
image and the assumed-movement pattern 1605 are displayed 
while they relatively move. That is, as for the 
driving-operation assist of this embodiment, if a 
procedure is executed under the same condition as the case 
of the driving-operation assist of the second embodiment, 
the synthetic image shown in FIG. 16(c) becomes same as 
the synthetic image shown in FIG. 14(c). 

According to this embodiment, because a 
circumferential-state image viewed from a viewpoint of 
a virtual camera moves in accordance with movement of an 
actual vehicle and the superimposed and synthesized 
assumed-movement pattern 1605 also moves in accordance 
with the movement of the vehicle. Therefore, both show 
coincident movement . Because a driver only has to operate 
a steering wheel along trace data for an assumed-movement 
pattern displayed at each point of time, a simpler and 
safer vehicle operation is realized. 

( Fourth embodiment ) 

Then, a fourth embodiment of the present invention 
will be described below by referring to the accompanying 
drawings . 

FIG. 17 is a block diagram showing a configuration 
of the driving-operation assist of the fourth embodiment 
of the present invention. The driving-operation assist 
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of this embodiment is also mainly used to assist driving 
operations for garaging, parallel parking and the like. 
Therefore, it is assumed that a component with no 
explanation is the same as the case of the first embodiment 
and a component provided with the same symbol as the case 
of the first embodiment has the same function as the case 
of the first embodiment unless otherwise explained. 
Moreover, it is assumed that each modified example 
described for the first embodiment can be applied to this 
embodiment by applying the same modification unless 
otherwise described. 

As shown in FIG. 17, the configuration of the 
driving-operation assist of this embodiment is different 
from that of the driving-operation assist of the first 
embodiment in that start inputting means 1701, driving 
controlling means 1702, integrating means 1703, and space 
converting means 1704 are included. 

The start inputting means 1701 is used for a driver 
to input and designate start of an actual driving operation 
(parking operation) corresponding to an assumed-movement 
pattern, which corresponds to operation start detecting 
means of the present invention. 

The driving controlling means 1702 automatically 
controls operations of a vehicle by controlling a 
steering-wheel rotation angle and the number of 
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revolutions of rear wheels in accordance with time-series 
data (e.g. FIG. 5(b)) corresponding to an assumed-movement 
pattern when a driving-operation start instruction is 
input, which corresponds _to operation controlling means 
of the present invention. 

The integrating means 1703 integrates a 
steering-wheel rotation angle and the number of 
revolutions of rear wheels and computes a spatial movement 
change of a vehicle up to the current point of time after 
a driving operation (parking operation) is started, which 
corresponds to moving-position computing means of the 
present invention. That is, the integrating means 1703 
has the same function as the integrating means 1302 in 
FIG. 13 described for the second embodiment. 

The space converting means 1704 moves an 
assumed-movement pattern in accordance with the spatial 
movement change and an assembly constituted of the 
superimposing means 102 and space converting means 1704 
corresponds to synthetic-image generating means of the 
present invention. That is, the space converting means 
1704 has the same functions as the space converting means 
1303 in FIG. 13 described for the second embodiment. 

In this embodiment, a procedure until a 
circumferential-state image of the present invention is 
generated in accordance with video data imaged by the 



imaging section 101 is the same as that described for the 
first embodiment. Moreover, a procedure until an actual 
driving operation corresponding to an assumed-movement 
pattern is started in a procedure until the superimposing 
means 102 generates a synthetic image of the present 
invention and the displaying means 107 displays the 
synthetic image is the same as that described for the first 
embodiment . 

A procedure until the superimposing means 102 
generates a synthetic image of the present invention and 
the displaying means 107 displays the synthetic image after 
an actual driving operation corresponding to an 
assumed-movement pattern is started is described below 
by using a case of performing parallel parking to the left 
side as an example. 

A driver locates a vehicle at a position suitable for 
start of a parking operation while viewing a synthetic 
image displayed on the displaying means 107 before starting 
the parking operation and then, inputs an instruction for 
start of the parking operation to the start inputting means 
1701. The start inputting means 1701 communicates that 
the driving-operation start instruction is input to the 
driving controlling means 1702 and integrating means 1703 
through the assumed-movement-pattern storing means 108. 
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When receiving a parking-operation start instruction, 
the driving controlling means 1702 automatically controls 
driving of a vehicle by generating a 
steering-wheel-rotation-angle control signal and a 
number-of-revolution-of-rear-wheel control signal in 
accordance with the time-series data (FIG. 5(b)) 
corresponding to an assumed-movement pattern and 
controlling a steering-wheel control system and a 
rear-wheel control system. 

In case of receiving a parking-operation start 
instruction, the integrating means 1703 integrates a 
steering-wheel-operation-angle signal and a 
number-of-revolution-of -rear-wheel signal to compute the 
positional relation between the current vehicle position 

1402 and the parking-operation start position 1401 as shown 
in FIG. 14(a) . 

As shown in FIG. 14(b), the space converting means 
1704 moves the assumed-movement pattern 1403 corresponding 
to parallel parking to the left side so that the operation 
start position (501 in FIG. 5) and the parking-operation 
start position 1401 coincide with each other in accordance 
with the computed positional relation. In other words, 
after start of a driving operation, the space converting 
means 1704 spatially fixes the assumed-movement pattern 

1403 to the parking-operation start position. 
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A subsequent procedure in which the superimposing 
means 102 generates a synthetic image and the displaying 
means 107 displays the synthetic image is the same as that 
described for the second, embodiment, when the vehicle 
reaches a predetermined parking position, the driving 
control means 1702 stops the vehicle in accordance with 
the time-series data and thereby, the parking operation 
is completed. 

According to this embodiment, it is possible to obtain 
an advantage that a steering-wheel operation, and the like 
are automatically performed in addition to the advantages 
described for the second embodiment after start of a driving 
operation is designated. Because a driver only has to 
confirm that a steering-wheel operation is automatically 
generated in accordance with the trace data for a displayed 
assumed-movement pattern at each point of time and monitor 
that a new obstacle appears, a simpler and safer vehicle 
operation is realized. 

(Fifth embodiment) 

Then, a fifth embodiment of the present invention will 
be described below by referring to the accompanying 
drawings . 

FIG. 18 is a block diagram showing a configuration 
of the driving-operation assist of the fifth embodiment 
of the present invention. The driving-operation assist 
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of this embodiment is also mainly used to assist driving 
operations for garaging, parallel parking and the like. 
Therefore, it is assumed that a component with no 
explanation is the same as_the case of the first embodiment 
and a component provided with the same symbol as the case 
of the first embodiment has the same function as the case 
of the first embodiment unless otherwise explained. 
Moreover, each modified example described for the first 
embodiment can be applied to this embodiment by applying 
the same modification unless otherwise described. 

As shown in FIG. 18, the configuration of the 
driving-operation assist of this embodiment is different 
from that of the first embodiment in that trace correcting 
means 1801 is included. 

The trace correcting means 1801 corrects an 
assumed-movement pattern and time-series data in 
accordance with a driving-operation start position and 
an operation end position input from a driver, which 
corresponds to pattern correcting means of the present 
invention . 

In this embodiment, a procedure until a 
circumferential-state image of the present invention is 
generated in accordance with video data imaged by the 
imaging section 101 is the same as that described for the 
first embodiment. Moreover, a procedure until an 



assumed-movement pattern stored in the 

assumed-movement-pattern storing means 108 is displayed 
on a synthetic image by making the operation start position 
of the pattern coincide with the current vehicle position 
in a procedure until the superimposing means 102 generates 
a synthetic image of the present invention and the 
displaying means 107 displays the synthetic image is the 
same as that described for the first embodiment. 

A procedure until a driver corrects an 
assumed-movement pattern and time-series data by using 
the trace correcting means 1801 and the assumed-movement 
pattern and time-series data are displayed on a synthetic 
image after the assumed-movement pattern is displayed on 
the synthetic image is described below by using a case 
of performing garaging to the left side as an example. 

As shown in FIG. 19(a), a case is assumed in which 
as a result of moving a vehicle to a current position 1901 
where an assumed-movement-pattern-operation end position 
coincides with a target parking position 1902 in order 
to perform a garaging operation by assuming the target 
parking position 1902 as an operation end position so that 
a driver does not contact with an obstacle 1904 or 1905, 
it is found that a tire trace 1903 of the assumed-movement 
pattern interferes with the obstacle 1905. 
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When the assumed-movement-pattern storing means 108 
stores an assumed-movement pattern for a garaging 
operation to another left side, it is possible to study 
whether a parking operation is smoothly performed by 
selecting the assumed-movement pattern stored in the means 
108 by pattern selecting means (not illustrated) . However, 
when the means 108 does not store the assumed-movement 
pattern or when another assumed-movement pattern also 
interferes with an obstacle, a driver corrects the 
as sumed-movement pattern . 

First, the driver moves a pattern showing a vehicle 
present at the current position 1901 of a synthetic image 
(FIG. 19(a)) displayed on the displaying means 107 to a 
new operation start position 1906 by inputting of a 
numerical value, a pointer, or other means as shown in 
FIG. 19(b). 

When the new operation start position 1906 is 
determined, the trace correcting means 1801 obtains a new 
tire trace 1907 (refer to FIG. 19(c)) for a vehicle to 
move from the new operation start position 1906 to the 
target parking position 1902 and generates new 
assumed-movement patterns and time-series data 
corresponding to the new assumed-movement patterns. 

The superimposingmeans 102 adjusts an operation start 
position of the new assumed-movement pattern to the current 
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position 19 01 of a vehicle and generates a synthetic image 
as shown in FIG. 19(d) and the displaying means 107 displays 
the synthetic image. 

Therefore, a driver can park a vehicle to the target 
parking position 1902 by moving the vehicle so as to adjust 
an operation end position 1908 of the new assumed-movement 
pattern to the target parking position 1902 and then, 
starting a driving operation (parking operation) in 
accordance with the new assumed-movement pattern. 

It is permitted to store the generated new 
assumed-movement pattern and time-series data in the 
assumed-movement-pattern storingmeans 108 after updating 
an original assumed-movement pattern or to additionally 
store the new assumed-movement pattern and time-series 
data in the assumed-movement pattern storing means 108 
without updating the original assumed-movement pattern. 
Moreover, it is permitted not to store them as temporary 
matters assumed-movement pattern and time-series data in 
the means 108 . Furthermore, it is permitted that a driver 
selects renewed storage, additional storage, or 
non-storage each time. 

Moreover, for this embodiment 1 , it is described that 
an assumed-movement pattern to be renewedly stored or 
additionally stored in the assumed-movement-pattern 
storing means 108 is automatically obtained in accordance 
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with positions of a vehicle at start and end of driving 
input by a driver . However , it is also permitted to perform 
an actual driving operation, sample time-series data at 
that time for the steering angle of a steering wheel, the 
number of revolution of wheels and the like, and generate 
and store an assumed-movement pattern in accordance with 
the time-series data. 

According to this embodiment , it is possible to realize 
an extensible driving-operation assist compared to the 
driving-operation assist of the first embodiment. 

(Sixth embodiment) 

Then, a sixth embodiment of the present invention will 
be described below by referring to the accompanying 
drawings . 

FIG. 2 0 is a block diagram showing a configuration 
of the driving-operation assist of the sixth embodiment 
of the present invention. The driving-operation assist 
of this embodiment is also mainly used to assist driving 
operations for garaging, parallel parking and the like. 
Therefore, in this embodiment, it is assumed that a 
component with no explanation is the same as the case of 
the first embodiment and a component provided with the 
same symbol as the case of the first embodiment has the 
same function as the case of the first embodiment unless 
otherwise explained. Moreover, each modified example 



described for the first embodiment can be applied to this 
embodiment by applying the same modification unless 
otherwise described. 

As shown in FIG. 20,_ the configuration of the 
driving-operation assist of this embodiment is different 
from that of the driving-operation assist of the first 
embodiment in that CG-image synthesizing means 2001 is 
included. 

The CG-image synthesizing means 2001 stores 
three-dimensional data corresponding to the 
assumed-movement patterns stored in the 
assumed-movement-pattern storing means 108 and generates 
a three-dimensional (or two-dimensional) image adjusted 
to a viewpoint of a circumferential-state image, which 
corresponds to a part of the function of 
assumed-movement -pattern storing means of the present 
invention and a part of the function of synthetic-image 
generating means of the present invention. 

It is assumed that the viewpoint converting means 106 
can change positions of a viewpoint automatically or in 
accordance with an input from a driver. 

Moreover, as shown in FIG. 21(a), the 
assumed-movement-pattern storing means 108 stores 
positions of a plurality of virtual poles 2001[sic;2101] 
arranged on the tire traces 503 together with stored 
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assumed-movement patterns (operation start position 501, 
operation end position 5 02 , and tire traces 503). The 
CG-image synthesizing means 2001 previously generates and 
stores three-dimensional- data (refer to FIG. 21(b)) 
corresponding to the assumed-movement patterns in 
accordance with the data for the assumed-movement patterns 
and the virtual poles 2001 [ sic; 2 101 ] . 

In this embodiment, a procedure until a 
circumferential-state image of the present invention is 
generated in accordance with the video data imaged by the 
imaging section 101 is the same as that described for the 
first embodiment. Moreover, a procedure until an actual 
driving operation corresponding to an assumed-movement 
pattern is started in a procedure until the superimposing 
means 102 generates a synthetic image of the present 
invention and the displaying means 107 displays the 
synthetic image is the same as that described for the first 
embodiment . 

A procedure until the superimposing means 102 
generates a synthetic image of the present invention and 
the displaying means 107 displays the synthetic image after 
an actual driving operation corresponding to an 
assumed-movement pattern is started is described below 
by using a case of performing parallel parking to the left 
side as an example. 
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Before starting an actual driving operation, a driver 
changes a viewpoint of a circumferential-state image used 
by the viewpoint converting means 106 to a viewpoint for 
viewing the rear from the back of the vehicle concerned 
from a position just above the vehicle concerned. Or, 
when the viewpoint converting means 106 detects that an 
actual driving operation corresponding to an 
assumed-movement pattern is started, the viewpoint is 
automatically changed to the latter viewpoint. Specific 
means for detecting that an actual driving operation is 
started includes, for examples, means same as the start 
detecting means 1301 described for the second embodiment. 

Because viewpoints are changed, a 
circumferential-state image which is an output from the 
viewpoint converting means 106 becomes the image shown 
in FIG. 21(c). The CG-image synthesizing means 2001 
generates a CG image viewed from a viewpoint same as the 
viewpoint used by the viewpoint converting means 106 by 
adjusting the current position of the vehicle concerned 
to the operation start position 501. In this case, the 
CG image becomes the image shown in FIG. 21(d). 

The superimposing means 102 superimposes the CG image 
on a circumferential-state image and synthesizes them to 
generate a synthetic image of the present invention as 
shown in FIG. 21(e). The displaying means 107 displays 
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the synthetic image. Because the synthetic image is an 
image viewed from a viewpoint fixed on a vehicle, the whole 
image is displayedwhile moving relatively when the vehicle 
moves in case of FIG. 21<e). 

In this embodiment, because a driver can determine 
a parking start position while viewing the displayed image 
and confirming the relation between a virtual pole or an 
operation end position and an actual obstacle at a glance, 
the driver can perform a safe and secure driving operation. 

In the above explanation, the CG-image synthesizing 
means 2001 generates a CG image from a viewpoint same as 
the viewpoint used by the viewpoint converting means 106 
in real time. However, it is also permitted to use a 
configuration of previously generating a CG image viewed 
from a predetermined viewpoint position every 
assumed-movement pattern and storing these CG images . 

Moreover, in this embodiment, it is described that 
a circumferential- state image viewed from a virtual camera 
is generated similarly to the case of the first embodiment . 
However, when purposing only an image viewing the rear 
from the back of a vehicle without changing any viewpoint, 
it is also permitted to directly use an image imaged by 
a camera set to the viewpoint position concerned as a 
circumferential-state image. A configuration of a 
driving-operation assist in this case is shown by the block 
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diagram in FIG. 22. That is, the CG-image synthesizing 
means 2001 generates a CG image by obtaining the data for 
a viewpoint of a vehicle-mounted camera 2 201 from the camera 
parameter table 103. 

(Seventh embodiment) 

Then, a seventh embodiment of the present invention 
will be described below by referring to the accompanying 
drawings . 

FIG. 23 is a block diagram showing a configuration 
of the driving-operation assist of the seventh embodiment 
of the present invention. The driving-operation assist 
of this embodiment is also mainly used to assist driving 
operations for garaging, parallel parking and the like. 
Therefore, in this embodiment, it is assumed that a 
component with no explanation is the same as the case of 
the first embodiment and a component provided with the 
same symbol as the case of the first embodiment has the 
same function as the case of the first embodiment unless 
otherwise explained. Moreover, it is assumed that each 
modified example described for the first embodiment can 
be applied to this embodiment by applying the same 
modification unless otherwise described. 

As shown in FIG. 23, the configuration of the 
driving-operation assist of this embodiment is different 
from that of the driving-operation assist of the first 
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embodiment in that mapping means 2 301 and a mapping table 
2302 are included. 

The mapping means 2301 fast performs the processing 
for converting an image input from each camera of the 
imaging section 101 into an image viewed from an optional 
viewpoint . 

The mapping table 2302 stores the data used for the 
mapping means 2301 to perform conversion. 

FIG. 24 is a conceptual view showing are example of 
a mapping table stored in the mapping table 2302. The 
mapping table is configured by cells equal to the number 
of pixels of a screen ( that is , a synthetic image generated 
by the superimposing means 102) displayed by the displaying 
means 107. That is, the mapping table is configured so 
that the number of horizontal pixels on a display screen 
becomes equal to the number of columns of the table and 
the number of vertical pixels on the display screen becomes 
equal to the number of rows of the table. Moreover, each 
cell has a camera number and pixel coordinates of an image 
photographed by each camera as data. 

For example, the top-left cell (cell in which the data 
(1, 10, 10) is entered) in FIG. 2 4 shows the top left, 
that is, the portion of (0,0) on the display screen and 
the mapping means 2301 performs the processing of 
d isplaying the data for the pixel (10,10) of an image 



- 51 - 

photographed by a camera No. 1 on a display screen (0,0)" 
in accordance with the data content (1,10,10) stored in 
the cell . By using the table, it is unnecessary to perform 
the computation for deciding the data for a pixel of a 
camera with which each pixel on a display screen should 
be replaced whether to replace the data for a pixel of 
a replaced one. Therefore, it is possible to realize to 
accelerate the processing. 

When the viewpoint converting means 106 uses a 
plurality of viewpoints by changing them, it is necessary 
that the table shown in FIG. 24 is stored in the mapping 
table 2302 every viewpoint. 

(Eighth embodiment) 

Then, an eighth embodiment of the present invention 
will be described below by referring to the accompanying 
drawings . 

FIG. 25 is a block diagram showing a configuration 
of the driving-operation assist of the eighth embodiment 
of the present invention. The driving-operation assist 
of this embodiment is also mainly used to assist driving 
operations for garaging, parallel parking and the like. 
Therefore, in this embodiment, it is assumed that a 
component with no explanation is the same as the case of 
the first embodiment and a component provided with the 
same symbol as the case of the first embodiment has the 
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same function as the case of the first embodiment unless 
otherwise explained. Moreover, it is assumed that each 
modified example described for the first embodiment can 
be applied to this embodiment by applying the same 
modification unless otherwise described. 

As shown in FIG. 25 , the configuration of the 
driving-operation assist of this embodiment is different 
from that of the driving-operation assist of the first 
embodiment in that final-position inputting means 2501, 
start-position determining means 2502, and space fixing 
means 2503 are included. 

The final-position inputting means 2501 inputs a 
target end position of a driving operation by a pointer. 
It is also permitted to input the target end position by 
inputting a numerical value or through other means. 

The start -posit ion determining means 2502 obtains a 
start position of a driving operation corresponding to 
a target end position input by the final-position inputting 
means 2501 in accordance with an assumed-movement pattern 
corresponding to the driving operation concerned. 

The space fixing means 2503 makes the target end 
position coincide with the operation end position and 
thereafter, spatially fixes an assumed-movement pattern 
corresponding to the driving operation, which has the 
functions of the integrating means 1302 and space 
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converting means 1303 in FIG. 13 (in FIG. 25, a 
rear-wheel-rotation-signal input and a 
steering-wheel-turning-angle signal input are not 
illustrated) . 

Moreover, it is permitted that the means 2503 has the 
functions of the image tracking means 1502 and space 
converting means 1503 in FIG. 15. In this case, however, 
it is necessary to receive an input of space data from 
the space buffer 105 [sic; space data buff er 105 ] similarly 
to the case of the image tracking means 1502 in FIG. 15. 
An assembly configured by the superimposing means 102 and 
space fixing means 2503 corresponds to synthetic-image 
generating means of the present invention. 

In this embodiment, a procedure until a 
circumferential-state image of the present invention is 
generated in accordance with the video data imaged by the 
imaging section 101 is the same as that described for the 
first embodiment. Moreover, a procedure until an 
assumed-movement pattern stored in the 
assumed-movement-pattern storing means 108 is displayed 
on a synthetic image by adjusting an operation start 
position of the pattern to a current position of a vehicle 
in a procedure until the superimposing means 102 generates 
a synthetic image of the present invention and the 
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displaying means 107 displays the synthetic image is the 
same as that described for the first embodiment. 

A procedure until a driver inputs a target end position 
of a driving operation by using the final-position 
inputting means 2501 and an assumed-movement pattern 
including a driving-operation start position 
corresponding to the target end position is displayed on 
the synthetic image after an assumed-movement pattern is 
displayed on a synthetic image will be described below 
by using a case of performing garaging to the left side 
as an example. 

As shown in FIG. 26(a), a case is assumed in which, 
as a result of driver's displaying a synthetic image of 
the present invention on the displaying means 107 to park 
a vehicle between the obstacles 401 and 4 02 so as not to 
contact them, the parking position 1001 which is the 
operation end position of the assumed-movement pattern 
1403 using the current position 901 of a vehicle as an 
operation start position overlaps with the obstacle 402. 

The driver moves the parking position 1001 to a target 
position 2602 by using a pointer 2 601 displayed on the 
screen of the displaying means 107 . In this case, as shown 
in FIG. 26(b), the assumed-movement pattern 1403 moves 
together with the parking position 1001. Therefore, the 
operation start position of the assumed-movement pattern 



1403 is displayed as a start position 2603 for starting 
a parking operation. 

As shown in FIG. 26(c), even after the above moving 
operation is completed, the current position 901 of the 
vehicle is still displayed on the screen of the displaying 
means 107. The driver only has to move the vehicle to 
the start position 2603 while viewing the screen. In this 
case, because the assumed-movement pattern 1403 is fixed 
in a space by the space fixing means 2 503, the relative 
positional relation between the assumed-movement pattern 
1403 and the obstacles 401 and 402 is not changed. 

According to this embodiment, because a 
driving-operation start position can be efficiently 
obtained in addition to the advantages described for the 
first embodiment, it is possible to decrease the time 
required by start of the operation. 

Moreover, it is permitted to add a start -position 
guiding means to the driving-operation assist of this 
embodiment, which computes a relative positional relation 
with the current position 901 when the start position 26 03 
is determined, obtains time-series data for a 
steering-wheel rotation angle and the number of 
revolutions of rear wheels necessary to guide a vehicle 
from the current position 901 up to the start position 
2603, generates a steering-wheel-rotation-angle control 
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signal and a number-of-revolution-of-rear-wheel control 
signal in accordance with the time-series data, controls 
a steering-wheel system and a rear-wheel driving system, 
and thereby automatically controls driving of the vehicle 
and automatically guides the vehicle from the current 
position 901 up to the start position 2603. Thereby, 
because the vehicle is guided up to a start position without 
operations by the driver, simpler and safer vehicle 
operations are realized. 
(Ninth embodiment) 

Then, a ninth embodiment of the present invention will 
be described below by referring to the accompanying 
drawings . 

FIG. 27 is a block diagram showing a configuration 
of the driving-operation assist of the ninth embodiment 
of the present invention. The driving-operation assist 
of this embodiment is different from that of the second 
embodiment only in that outputs of the start detecting 
means 1301 and the integrating means 1302 are input to 
the viewpoint converting means 106 and the viewpoint 
converting means 106 changes viewpoints of a virtual camera 
in accordance with the outputs. 

Therefore, in this embodiment, it is assumed that a 
component with no explanation is the same as the case of 
the second embodiment and a component provided with the 
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same symbol as the case of the second embodiment has the 
same function as the case of the second embodiment unless 
otherwise explained. Moreover, it is assumed that each 
modified example described for the second embodiment can 
be applied to this embodiment by applying the same 
modification unless otherwise described. 

In this embodiment, a procedure until a 
circumferential-state image of the present invention is 
generated in accordance with the video data imaged by the 
imaging section 101 is the same as that described for the 
first embodiment. Moreover, a procedure until an actual 
driving operation corresponding to an assumed-movement 
pattern is started in a procedure until the superimposing 
means 102 generates a synthetic image of the present 
invention and the displaying means 107 displays the 
synthetic image is the same as that described for the first 
embodiment . 

A procedure until the superimposing means 102 
generates a synthetic image of the present invention and 
the displaying means 107 displays the synthetic image after 
an actual driving operation corresponding to an 
assumed-movement pattern is started will be described 
below by using a case of performing parallel parking to 
the left side as an example. 
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Before an actual driving operation is started, a 
viewpoint position of a virtual camera is fixed to a 
position just above a vehicle as described for the first 
embodiment by referring to FIG. 10, the current vehicle 
position 9 01 and the assumed-movement pattern 1403 using 
the position 901 as an operation start position is fixed 
on a screen as shown in FIG. 28(a), and 
circumferential-state images such as the obstacles 401 , 
402 or the like, are displayed while relatively moving 
on the screen in accordance with movement of the vehicle. 

As shown in FIG. 28(b), when the current vehicle 
position 901 comes to a position corresponding to the target 
parking position 902, a driver starts a driving operation 
corresponding to parallel parking to the left side. When 
the driving operation is started, a gear signal shows a 
backward-movement state and a front-wheel steering angle 
according to a steering-wheel-turning-angle signal 
reaches a certain value or more. Therefore, the start 
detecting means 1301 judges that the driving operation 
(parking operation) corresponding to the parallel parking 
to the left side is started and communicates that the 
driving operation (parking operation) is started to the 
integrating means 1302 and viewpoint converting means 106. 

The integrating means 1302 integrates input 
steering-wheel-rotation-angle signal and 
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number-of-revolution-of -rear-wheel signal after start of 
the driving operation to compute the positional relation 
between the current vehicle position 1402 and the 
parking-operation start position 1401 as shown in FIG. 
14(a). 

In accordance with the computed positional relation, 
the space converting means 1303 moves the assumed-movement 
pattern 1403 corresponding to the parallel parking to the 
left side so that the operation start position (501 in 
FIG. 5) of the pattern 1403 coincides with the 
parking-operation start position 1401 as shown in FIG. 
14(b). In other words, the space converting means 1303 
spatially fixes the assumed-movement pattern 1403 to the 
position at start of the parking operation after start 
of the driving operation. 

When the viewpoint converting means 106 receives that 
the driving operation (parking operation) is started, it 
fixes the then viewpoint position of the virtual camera 
to a space (ground) . That is, after start of the driving 
operation, the circumferential-state images (obstacles 
401, 402 or the like) are fixed on the screen. 

After start of the driving operation, the 
superimposing means 102 superimposes the assumed-movement 
pattern 1403 spatially fixed to the position at start of 
the parking operation and the current vehicle position 
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1402 on the circumferential-state images and synthesizes 
them to generate a synthetic image of the present invention 
as shown in FIG. 28(c). Because the viewpoint of the 
synthetic image is obtained by spatially fixing the 
viewpoint position of the virtual camera at start of the 
parking operation similarly to viewpoints of 
circumferential-state images. Therefore, the 
superimposing means 102 generates the synthetic image by 
computing the positional relation computed by the 
integrating means 1302 inversely to a viewpoint. That 
is, in this synthetic image (FIG. 28(c)), the 
circumferential-state images (obstacles 401, 402 or the 
like) and the assumed-movement pattern 1403 are fixed on 
the screen and the current vehicle position 1402 is 
displayed while relatively moving on the screen in 
accordance with actual movement of the vehicle. 

According to this embodiment, because a viewpoint 
after start of a parking operation is fixed in a space, 
a driver can confirm a vehicle moving state to a 
circumferential state of a parking space at a glance. 

When a movement assumed in accordance with an 
assumed-movement pattern includes a wide range or is 
complex and a range larger than the field of view of a 
camera of the imaging section 101 occurs after a driving 
operation is started, it is also permitted to display the 



- 61 - 

range by using the data stored in the space-data buffer 
105. 

(Tenth embodiment) 

Then, a tenth embodiment of the present invention will 
be described below by referring to the accompanying 
drawings . 

FIG. 29 is a block diagram showing a configuration 
of the driving-operation assist of the tenth embodiment 
of the present invention. The driving-operation assist 
of this embodiment is different from that of the first 
embodiment only in that two cases are assumed in which 
a vehicle moves backward when turning a steering wheel 
up to the maximum angle clockwise and counterclockwise 
to tire rotation as default driving patterns stored in 
the assumed-movement -pattern storing means 108 as shown 
in FIG. 30(b) , two circumscribed area traces 604 in a space 
through which the whole of the vehicle passes in stead 
of the tire traces 603 caused by assumed movements of the 
two cases are synthesized by the superimposing means 102 
at the same time as shown in FIG. 30(a) , and the synthesized 
result is displayed on the displaying means 107. 

Therefore, in this embodiment, it is assumed that a 
component with no explanation is the same as the case of 
the first embodiment, a component provided with the same 
symbol as the case of the first embodiment has the same 
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function as the case of the first embodiment unless 
otherwise explained. Moreover, it is assumed that each 
modified example described for the first embodiment can 
be applied to this embodiment by applying the same 
modification unless otherwise described. 

According to this embodiment, two simplest cases are 
assumed in which a vehicle moves backward when turning 
a steering wheel up to the maximum angle clockwise and 
counterclockwise as shown in FIG. 30(b) as defaults of 
the assumed-movement storing pattern 108 [sic; 
assumed-movement pattern storing means]. Therefore, 
when the vehicle can be parked in accordance with the 
defaults of the assumed-movement storing pattern 108 [ sic ; 
assumed-movement pattern storing means], advantages can 
be obtained that it becomes unnecessary to change and select 
a plurality of assumed-movement storing patterns and an 
operating load of a driver can be reduced. 

Moreover, it becomes unnecessary to change and select 
two left and right assumed-movement storing patterns by 
simultaneously synthesizing circumscribed area traces 604 
produced due to assumed movements of the above described 
two cases by the superimposing means 102 and by displaying 
the synthesized result on the displaying means 107, and 
consequently there can be obtained such an effect that 
an operating load of the driver can be reduced. 
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Furthermore, as shown in FIG. 30(a) , the circumscribed 
area traces 60 in a space through which the whole of the 
vehicle passes instead of the tire traces 603 are 
synthesized by the superimposing means 102, and the 
synthesized result is displayed on the displaying means 
107, and thereby an advantage can be obtained that a driver 
can easily and more accurately confirm whether a portion 
of a vehicle to be protruded beyond a tire trace such as 
the right front portion of the vehicle when parking the 
vehicle while turning left-backward contacts with an 
obstacle. 

(Eleventh embodiment) 

Then, an eleventh embodiment of the present invention 
will be described below by referring to the accompanying 
drawings . 

FIG. 31 is a block diagram showing a configuration 
of the driving-operation assist of the eleventh embodiment 
of the present invention. The driving-operation assist 
of this embodiment is also mainly used to assist driving 
operations for garaging and parallel parking. Therefore, 
in this embodiment, it is assumed that a component with 
no explanation is the same as the case of the first and 
fifth embodiments and a component provided with the same 
symbol as the case of the first and fifth embodiments has 
the same function as the case of the first and fifth 
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embodiments unless otherwise explained. Moreover, it is 
assumed that each modified example described for the first 
and fifth embodiments can be applied to this embodiment 
by applying the same modification unless otherwise 
described. 

The configuration of the driving-operation assist of 
this embodiment is different from those of the first and 
fifth embodiments in that the obstacle inputting means 
3101 and assumed-pattern correcting means 3102 shown in 
FIG. 31 are included arid the assumed-movement-pattern 
storing means 108 includes a movement pattern for changing 
backward and forward movements of a vehicle while the 
vehicle moves as shown in FIG. 32(a). 

In this case, a steering-wheel angle corresponding 
to tire rotation is stored in the assumed-movement-pattern 
storing means 108 in FIG. 31 as time-series data for 
assumed-movement patterns as shown in FIG. 32 (b) . As shown 
in FIG. 32(b), tire rotation of 0 to 0.8 shows backward 
movement of a vehicle at which backward movement is changed 
to forward movement. In this case, the position of the 
vehicle is present at a backward- and forward-movement 
changing position 32 01 shown in FIG. 32(a). Then, the 
vehicle moves forward to a tire angle of 0.8 to 0.6. When 
the vehicle reaches the forward- and backward-movement 
changing position 3202 shown in FIG. 32(a), forward 
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movement is changed to backward movement again such as 
a tire angle of 0.6 to 1.4. 

Thus, by including a movement pattern for changing 
backward and forward movements during movement, it is 
possible to control a position and direction of a vehicle 
even if there is only a small spatial margin for an obstacle 
as shown in FIG. 32(a). 

In this embodiment, a procedure until a 
circumferential-state image of the present invention is 
generated in accordance with the video data imaged by the 
imaging section 101 is the same as that described for the 
first embodiment. A procedure until an assumed-movement 
pattern stored in the assumed-movement-pattern storing 
means 108 is displayed on a synthetic image of the present 
invention by adjusting an operation start position of the 
pattern to the current position of a vehicle in a procedure 
until the superimposing means 102 generates the synthetic 
image and the displaying means 107 displays the synthetic 
image is the same as that described for the first 
embodiment . 

A procedure until a driver corrects an 
assumed-movement pattern and time-series data by using 
the trace correcting means 1801 and obstacle inputting 
means 3101 after the assumed-movement pattern is displayed 
on a synthetic image and the corrected assumed-movement 
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pattern and time-series data are displayed on the synthetic 
image is described below by using a case of performing 
garaging to the right side as an example. 

As shown in FIG. 33 , a-case is assumed in which a driver 
moves a vehicle to the current position 1901 where an 
operation end position of an assumed-movement pattern 
coincides with the target parking position 1902 in order 
to perform a garaging operation using the target parking 
position 1902 as an operation end position so that the 
vehicle does not contact with an obstacle (a 3204, b 3205, 
or c 3206) and thereby, it is found that the circumscribed 
area 604 of the assumed-movement pattern may contact with 
the obstacle a, b, or c. 

When the assumed-movement-pattern storing means 108 
stores an assumed-movement pattern for another garaging 
operation to the right side, it is permitted to select 
the assumed-movement pattern by pattern selecting means 
(not illustrated) and study if a parking operation can 
be smoothly performed. However, when the means 108 does 
not store the assumed-movement pattern or another 
assumed-movement pattern also interferes with an obstacle 
or the like, the driver corrects the assumed-movement 
pattern . 

First, the driver assumes a pattern showing the vehicle 
present at the current position 32 00 of the synthetic image 
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(FIG. 33) shown on the displaying means 107 in FIG. 31 
as the parking-operation start position 1901 , assumes an 
area in the image in which the obstacle a, b, or c is present 
as an obstacle designation rectangle: a 3207 as shown in 
FIG. 34 by using the obstacle inputting means 3101 in FIG. 
31 or as an obstacle designation area: 3210 (refer to FIG. 
36) by using an obstacle designation circle: b 3208, and 
sets and inputs the obstacle designation rectangle or the 
obstacle designation area through numerical-value input, 
a pointer, or other means. Moreover, when it is necessary 
to correct the target parking position , the driver performs 
movement input through numerical-value input, a pointer, 
or other means . 

When an obstacle area is input, the trace correcting 
means 1801 sets a contact-hazard area 3209 in an area having 
a circumference of 60 cm including the obstacle designation 
area: 3210 as shown in FIG. 35. Moreover, a contact-hazard 
evaluation function H3210 as shown in FIG. 36 is provided 
for the area. The function is configured by synthesizing 
three quadratic functions which sharply increase as 
approaching for 10 cm or less from the obstacle designation 
area: 3210, slowly decrease as separating up to 10 cm or 
more, and become 0 for 60 cm or more. 

Moreover, as shown in FIG. 37, an evaluation point 
3211 (xi,yi) : (i=l-6) is set to six points around a certain 
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area as shown in FIG. 37 and a trace evaluation point 3212 
(xi,yi)n: (n=l-N) about N items (tire rotation tm and tire 
angle km) : (m=l-N) in the table shown in FIG. 32(b) is 
computed. 

As shown by the expression in FIG. 38, a 
trace-contact-hazard evaluation function H" 3213 is 
obtained from the total of the contact-hazard evaluation 
function H3210 at the position of the trace evaluation 
point 3212. 

The trace-contact-hazard evaluation function H" 3213 
becomes a function of N items (tire rotation tm and tire 
angle km) in the table in FIG. 32(b) as shown in FIG. 37. 
Therefore, by successively correcting the diagram (tire 
rotation tm and tire angle km) through the partial 
differential method, it is possible to obtain values (tire 
rotation tm and tire angle km) for minimizing the 
trace-contact-hazard evaluation function H" 3213. 

Thereby, as shown in FIG. 39 , it is possible to correct 
an assumed-movement pattern from (tire rotation tm and 
tire angle km) first shown in FIG. 32(b) to (tire rotation 
tm and tire angle km) for minimizing the 
trace-contact-hazard evaluation function H" 3213. 

When even one point of 10 cm or less is found in the 
obstacle designation area: 3210 from the contact-hazard 
evaluation function H about each point of trace evaluation 
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points 3212 (xi,yi)n in (tire rotation tm and tire angle 
km) for minimizing the trace-contact-hazard evaluation 
function H" 3213, a warning C areful" is issued to a driver . 
When even one point of 0 cm or less is found in the obstacle 
designation area: 3210, a warning Parking not possible" 
is issued to a driver. 

Because the corrected assumed-movement pattern is a 
movement pattern for minimizing the trace-contact-hazard 
evaluation function H" 3213, the assumed-movement pattern 
generates a trace having a larger margin from an obstacle 
as shown in FIG. 40 and thereby, a vehicle can be parked 
more safely. 

As shown in FIG. 41, the superimposing means 102 
generates a synthetic image by adjusting the operation 
start position 1901 of a corrected assumed-movement 
pattern 3214 to a current vehicle position 3200 and the 
displaying means 107 displays the synthetic image. 

Therefore, when a driver starts a driving operation 
(parking operation) of a vehicle in accordance with the 
new corrected assumed-movement pattern 3214, the driver 
can park the vehicle at the target parking position 1902 
in accordance with a movement pattern having a larger margin 
from an obstacle. 

It is permitted to store generated new 
assumed-movement pattern and time-series data in the 
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assumed-movement-pattern storing means 108 by updating 
an original assumed-movement pattern or to additionally 
store the new assumed-movement pattern in the means 108 
without changing the original assumed-movement pattern. 
Moreover, it is permitted to temporarily use them without 
storing them. Furthermore, it is permitted that a driver 
selects renewed storage, additional storage, or 
non-storage each time. 

Furthermore, this embodiment is described above by 
assuming that an assumed-movement pattern to be 
renewedly-stored or additionally-stored in the 
assumed-movement -pattern storing means 108 can be 
automatically obtained in accordance with a position of 
a vehicle at start or end of a movement input from a driver. 
However, it is also permitted to perform an actual driving 
operation and thereby, sample time-series data for the 
then steering angle of a steering wheel and the number 
of revolutions of wheels, generate an assumed-movement 
pattern in accordance with the time-series data, and store 
the pattern. 

(Twelfth embodiment) 

Then, a twelfth embodiment of the present invention 
will be described below by referring to FIGS. 42 to 46. 

In the eleventh embodiment, it is assumed that an 
assumed-movement pattern shown in FIG. 32(a) is corrected 
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in accordance with an obstacle area input from a driver. 
However, it is also permitted to correct two 
assumed-movement patterns or more and select a preferable 
one. FIG. 42 is different from FIG. 31 in that 
movement-pattern selection trace correcting means 3301 
is used instead of the trace correcting means 3101. 

First, operations of the twelfth embodiment are 
described. As shown in FIG. 43, a driver designates and 
inputs the target parking position 1902 on an image 
displayed on a display unit by assuming a current vehicle 
position as a parking-operation start position. The 
movement-pattern selection trace correcting means 3301 
extracts two assumed-movement patterns for parking a 
vehicle right backward as shown in FIG. 44 in accordance 
with a rough positional relation of the target parking 
position 1902 to the parking-operation start position out 
of a plurality of assumed-movement patterns stored in the 
assumed-movement-pattern storing means 108. 

Moreover, as shown in FIGS. 45 and 46, when the driver 
inputs an obstacle area on the image displayed on the 
display unit, the two assumed-movement patterns for 
respectively minimizing the trace-contact-hazard 
evaluation function H" 3213 are corrected similarly to 
the case of the eleventh embodiment. In this case, by 
comparing the two minimized trace-contact-hazard 
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evaluation functions H" 3213 with each other and selecting 
a smaller one of the two functions H" 3213, it is possible 
to select a safer assumed-movement pattern. Moreover, 
when the two trace-contact-hazard evaluation functions 
H" 3213 have the same value, a simpler assumed-movement 
pattern is selected by previously raising the priority 
of a function H" 3213 allowing a simpler driving operation. 

Therefore, when a spatial margin from an obstacle is 
small, a safer movement pattern having a cutback is 
automatically selected as shown in FIG. 45 . However, when 
a spatial margin from an obstacle is large, a simpler 
movement pattern having no cutback is automatically 
selected as shown in FIG. 446 [sic; Fig. 46]. 

According to the above configuration, a driver can 
automatically select an optimum assumed-movement pattern 
only by inputting a target parking position and an obstacle 
area and thereby realize optimum parking through a safer 
and simpler driving operation. 

According to this embodiment, it is possible to realize 
an extensible driving-operation assist compared to the 
driving-operation assist of the first embodiment. 

The above first to twelfth embodiments are described 
by assuming that circumferential-state imaging means of 
the present invention mainly generates an image viewed 
from a viewpoint of a virtual camera by using a plurality 
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of vehicle-mounted cameras . However , it is also permitted 
to use one camera set to the ceiling of a roof-provided 
parking lot. In short, it is possible to use any type 
of means as circumferential-state imaging means of the 
present invention as long as the means generates a 
circumferential-state image by imaging a circumferential 
state of a vehicle with a camera and/ or stores the generated 
circumferential-state image. 

Moreover, the above first to twelfth embodiments are 
described by assuming that driving-operation assist of 
the embodiments are respectively provided with 
as sumed-movement-pattern storing means of the present 
invention. However, it is also permitted to use any type 
of means as assumed-movement-pattern storing means of the 
present invention as long as an assumed-movement pattern 
can be input to the means each time. In short, it is 
permitted to use any type of driving-operation assist as 
driving-operation assist of the present invention as long 
as the driving-operation assist includes 
circumferential-state imaging means for imaging a 
circumferential state of a vehicle with a camera and thereby 
generating a circumferential-state image and/or storing 
the circumferential-state image, synthetic-image 
generating means for superimposing on the 
circumferential-state image, an assumed-movement pattern 



which is video data showing movement of the vehicle in 
the case of performing a predetermined series of driving 
operations for the vehicle and generating a synthetic image, 
and displaying means for displaying the synthetic image. 
Thereby, because at least a circumferential-state image 
and an assumed-movement pattern are superimposed on each 
other and displayed on a synthetic screen, a driver can 
confirm the relation between the circumferential-state 
image and the assumed-movement pattern at a glance. 

Industrial Applicability 

As apparent from the above description, the present 
invention provides a driving-operation assist by which 
a driver can directly confirm the relation between vehicle 
movement according to a predetermined series of driving 
operations and a circumferential state by displaying the 
vehicle movement when the driver performs the 
predetermined series of driving operations together with 
the circumferential sate and a driver ' s load can be reduced . 

That is, a driver can confirm a position for starting 
a driving operation such as garaging or parallel parking, 
a place for finally stopping a vehicle, and a positional 
relation with an obstacle such as other vehicle by the 
display image at a glance by using a driving-operation 
assist of the present invention. Therefore, it is expected 
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that a driver's operation load is reduced and safety is 
improved. Moreover, by introducing an automatic -driving 
technique, it is possible to automatically perform every 
operation including garaging only by driver's moving a 
vehicle up to a parking-operation start position while 
viewing the movement trace data. 

Furthermore, the present invention provides a 
recording medium for storing a program for making a computer 
execute all or some of functions of each means of a 
driving-operation assist of the present invention. 



CLAIMS 



1. A driving-operation assist characterized by 
comprising: 

circumferential-state imaging means for imaging a 
circumferential state of a vehicle with a camera and 
generating a circumferential-state image and/or storing 
the generated circumferential-state image; 

synthetic-image generating means for generating a 
synthetic image by superimposing on the 
circumferential-state image, an assumed-movement pattern 
which is the video data showing movement of the vehicle 
in case of performing a predetermined series of driving 
operations for the vehicle; and 

displaying means for displaying the synthetic image. 

2. The driving-operation assist according to claim 
1, characterized in that 

the circumferential-state imaging means has one 
camera or more and a camera parameter table for storing 
a camera parameter which is a characteristic of the camera 
or each of the cameras and generates the 
circumferential-state image on the basis of the camera 
parameter from an output of the camera or each of the 
cameras . 



3. The driving-operation assist according to claim 

2 , wherein 

the circumferential-state imaging means has space 
reconfiguring means for generating space data obtained 
by relating each pixel constituting an image output from 
the camera or each of the cameras to a point in a 
three-dimensional space on the basis of the camera 
parameter, and viewpoint converting means for generating 
an image viewed from a predetermined viewpoint as said 
circumferential-state image by referring to the space data 
and the synthetic-image generating means generates the 
synthetic image by referring to the space data. 

4. The driving-operation assist according to claim 

3, characterized in that 

a space-data buffer for temporarily storing the space 
data is included. 

5. The driving-operation assist according to claim 
3 or 4, characterized in that 

the predetermined viewpoint is a point fixed to the 
three-dimensional space or the vehicle, and 

the viewpoint converting means changes the 
predetermined viewpoint automatically or through an input 
from a driver. 

6 . The driving-operation assist according to any one 
of claims 1 to 5, characterized in that 



- 78 - 

the assumed-movement pattern includes video data 
showing the relation between an assumed-movement start 
area which is an area in which the vehicle at start of 
the movement of the vehicle when performing the above 
predetermined series of driving operations is present and 
an assumed-movement end area which is an area in which 
the vehicle at end of the movement is present. 

7. The driving-operation assist according to claim 
6, characterized in that 

the assumed-movement pattern includes video data 
showing tire traces of the vehicle and/ or video data showing 
a movement area of the vehicle. 

8. The driving-operation assist according to claim 
6 or 7, characterized in that 

the assumed-movement pattern includes video data 
showing virtual poles arranged on the outer edge of the 
vehicle movement area. 

9 . The driving-operation assist according to any one 
of claims 6 to 8, characterized in that 

the synthetic-image generating means superimposes 
current-position data serving as video data showing an 
area in which the vehicle is present, on the 
circumferential-state image to generate the synthetic 
image . 
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10 . The driving-operation, assist according to claim 
9, characterized in that 

the synthetic-image generating means superimposes 
the assumed-movement start area on a position same as the 
current-position data. 

11. The driving-operation assist according to any 
one of claims 6 to 10, characterized in that 

when actual driving operations corresponding to the 
above predetermined series of driving operations are 
started, 

the synthetic -image generating means thereafter 
fixes the positional relation between the assumed-movement 
pattern and the circumf erential-state image at the point 
of time when the actual driving operations are started 
and generates the synthetic image. 

12 . The driving-operation assist according to claim 
11, characterized in that 

positional-information storing means is included 
which stores positional information of the whole or a part 
of the video data for the assumed-movement pattern with 
regard to the basis of the whole or a part of the video 
data for the circumf erential-state image on the synthetic 
image when the actual driving operations are started, 



- 80 - 

the synthetic-image generating means fixes the 
positional relation in accordance with the positional 
information . 

13 . The driving-operation assist according to claim 
11, characterized in that 

movement -posit ion computing means is included which 
computes movement positions of the vehicle since the actual 
driving operations were started, in accordance with 
signals relating to the actual driving operations, and 

the synthetic-image generating means fixes the 
positional relation in accordance with the movement 
positions. 

14. The driving-operation assist according to any 
one of claims 6 to 13, characterized in that 

final-position inputting means for inputting a final 
position which is a position of the vehicle at end of the 
movement and start-position determining means for 
obtaining a start position which is a position at start 
of the movement corresponding to the input final position 
in accordance with the assumed-movement pattern are 
included, and 

the synthetic-image generating means superimposes 
the input final position and the start position 
corresponding to the input final position on the 



circumf erential-state image to generate the synthetic 
image . 

15 . The driving-operation assist according to claim 
14, characterized in that 

start-position guiding means is included which guides 
the vehicle to the start position by automatically 
controlling driving of the vehicle. 

16. The driving-operation assist according to any 
one of claims 1 to 15, characterized in that 

assumed-movement-pattern storing means is included 
which holds data relating to the above predetermined series 
of driving operations and holds assumed-movement data 
including at least the assumed-movement pattern. 

17 . The driving-operation assist according to claim 
16, characterized in that 

the assumed-movement -pattern storing means holds a 
plurality of assumed-movement patterns, and 

pattern selecting means is included which 
automatically selects the assumed-movement pattern 
through an input from a driver or a predetermined driving 
operation . 

18. The driving-operation assist according to claim 
16 or 17, characterized in that pattern correcting means 
is included which is able to update and correct the content 
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of the assumed-movement pattern held in the 
assumed-movement-pattern storing means. 

19 . The driving-operation assist according to claim 
18 , characterized in that the pattern correcting means 
updates and corrects the assumed-movement pattern and/or 
the assumed-movement data in accordance with the vehicle 
positions at start and end of the corrected movement input 
from a driver. 

20 . The driving-operation assist according to claim 
18, characterized in that the pattern correcting means 
updates and corrects the assumed-movement pattern and/or 
the assumed-movement data in accordance with an actual 
driving operation. 

21. The driving-operation assist according to any 
one of claims 16 to 20 , characterized in that the 
assumed-movement data includes time- series data showing 
a relation between a movement distance and a steering angle 
of the steering wheel of the vehicle. 

22 . The driving-operation assist according to claim 
2 1 , characterized in that driving control means is included 
which automatically controls driving of the vehicle in 
accordance with the time-series data when actual driving 
operations corresponding to the above predetermined series 
of driving operations are started. 
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23. The driving-operation assist according to any- 
one of claims 11 to 13 or claim 22, characterized in that 
operation-start detecting means is included which 
automatically detects that actual driving operations 
corresponding to the above predetermined series of driving 
operations are started through an input from a driver or 
a predetermined driving operation. 

24. The driving-operation assist according to any 
one of claims 5 to 23, characterized in that 

when the viewpoint converting means changes the 
predetermined viewpoint, the viewpoint converting means 
fixes the predetermined viewpoint to the vehicle before 
the actual driving operations corresponding to the 
predetermined series of driving operations are started 
and changes the predetermined viewpoint to a point fixed 
to the three-dimensional space after the actual driving 
operations corresponding to the predetermined series of 
driving operations are started. 

25 . The driving-operation assist according to claim 
24, characterized in that the point fixed to the 
three-dimensional space is a point just above the position 
at end of the movement of the vehicle shown by the 
assumed-movement pattern when the actual driving 
operations are started. 
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26. The driving-operation assist according to any- 
one of claims 1 to 5, characterized in that the 
assumed-movement pattern includes video data showing a 
circumscribed area on a space through which the vehicle 
passes when the predetermined series of driving operations 
are performed. 

27 . The driving-operation assist according to claim 
6, characterized in that the assumed-movement pattern 
includes video data showing a trace when maximizing a 
steering angle of tires of the vehicle clockwise or 
counterclockwise and/or video data showing movement areas 
of the vehicle. 

28 . The driving-operation assist according to claim 
16, characterized in that the assumed-movement -pattern 
storing means holds the assumed-movement patterns and is 
provided with synthetic-image generating means for 
generating a synthetic image by simultaneously 
superimposing two or more of the assumed-movement patterns 
on the circumferential-state image and displaying means 
for displaying the synthetic image. 

29. The driving-operation assist according to any 
one of claims 1 to 5, characterized in that the 
assumed-movement patterns respectively include a change 
from backward movement to forward movement or from forward 
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movement to backward movement in the predetermined series 
of driving operations. 

30. The driving-operation assist according to any 
one of claims 1 to 5, characterized in that obstacle 
inputting means is included which is able to input a 
position of an obstacle area in an image to displaying 
means for displaying the synthetic image from a driver. 

31 . The driving-operation assist according to claim 

17, characterized in that 

the pattern selecting means selects one of the 
assumed-movement patterns in accordance with an 
obstacle-area position input from a driver, and 

obstacle inputting means is included which is able 
to input a position of an obstacle area in an image to 
displaying means for displaying the synthetic image from 
a driver. 

32 . The driving-operation assist according to claim 

18, characterized in that 

the pattern correcting means updates and corrects the 
assumed-movement patterns and/or the assumed-movement 
data in accordance with a position of an obstacle area 
input from a driver, and 

obstacle inputting means is included which is able 
to input a position of an obstacle in an image to displaying 
means for displaying the synthetic image from a driver. 
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33. A recording medium characterized by storing a 
program of making a computer execute all or some of 
functions of each means of any one of claims 1 to 32 . 



ABSTRACT 



A driving-operation assist, is characterized by 
comprising : 

circumferential-state imaging means for imaging a 
circumferential state of a vehicle with a camera and 
generating a circumferential-state image; 

synthetic-image generating means for generating a 
synthetic image by superimposing on the 

circumferential-state image, an assumed-movement pattern 
which is the video data showing movement of the vehicle 
in case of performing a predetermined series of driving 
operations for the vehicle; and 

displaying means for displaying the synthetic image, 

and 

the circumferential-state imaging means has one 
camera or more and a camera parameter table for storing 
a camera parameter which is a characteristic of the camera 
or each of the cameras and generates the 
circumferential-state image on the basis of the camera 
parameter from an output of the camera or each of the 
cameras . 



Q9/5S1004 



1/44 



00 
CD- 



CO 
cr 



u 




-P 




1 








to 




a. 




as 




u 


Q) 


a> 


rH 


9 


X) 




HJ 


U 


-P 



CM 



CU rtJ 

o a) 



CD 

OJ -M 





tn 






a 






■•-i 






M 






13 






CP 






-H 




0) 


m 
a 


Ui 


V 


o 


a 


as 


o 




Q4 


0) 




to 


u 


g 



LO 



tuo 



09/581004 



2/44 

Fig. 2 



Camera 5 

205 Camera 6 

v 206 201 Camera 1 




09/581004 



3/44 

Fig. 3 




09/581004 



4/44 



Circumferential-state image 



401 Obstacle 




402 Obstacle 




09/581004 

5/44 



! 0) 
- -p <— I 

! in a) 



LQ 

tuO 




C9 ,'581004 



6/44 




09/581004 



7/44 

7 



Synthetic image 




09/581004 



8/44 




09/581004 



9/44 

ig. 9 




402 Obstacle 



09/581004 



10/44 





99/581004 




09/581004 




09/581 QQ4 



13/44 



00 







M 




CD 




4-1 




4J 




fd 
P-i 




4-> 




c 








I 


en 
« 








<D 


§ 

1 


a 




tn 




CS 




■H 


I 


M 


w 


O 


w 


4-1 


<! 


CO 



CO 

o 

CO- 



o 

CO 



jo uo-rq.eq.oH " 
BTDUB uoxq.Bq.oj 

X93qA-BUTJB9q.S 











a 






-H 






+j 




4-> 


o 




M 


<u 


a 


rd 


+J 




4J 


a> 








I 



CO 

o 



4-» nj 



CM 
O 



a a? 
to S 



W > fi 

0) C nj 
-H O <D 







c 




•H 




>i 




nJ 




rH 


CO 


PU 




in 




-H 




a 


i 



CO 
fcuO 



09/581004 




Fig. 14 (c) 

Synthetic image 




09/581 004 



15/44 



CO 























03 




1 








CD 






ttl 


1 


a 




nj 


1 


CD 


1 


€ 




CP 




CS 


1 


-H 




M 


(0 


O 


W 


+J 







X33iIM-6iiTJ:aaq.s 



£ 2 

CO 6 



CO 
O 
LO ' 



(0 ro 
ft 02 



o 

LO 



LO 




CM 
O 



LO 
fcuO 



en 
-3 



ft "J 

CO S 











CP 






C3 
-H 
>i 
f0 






rH 


10 




ft 


G 




W 












O 


1 







CM 


















a> 






a 








o 




o 



09/581004 



16/44 



Fig. 16(a) 
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Fig. 30(b) 
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Fig. 32 (a) 



Circumscribed area 
(Solid line 

604 




Parking-operation 
start position 



3202 

Forward/backward-movement 
changing position (Broken 
line) 

3201 

Backward/ forward -movement 
changing position (Broken 
line) 



Target parking 
position 



Fig. 32 (b) 
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Fig. 38 
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Fig. 39 
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Fig. 40 
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Fig. 41 
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^Declaration and Power of Attorney For Patent Application 

English Language Declaration 

As a beiow named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name, 

I beiieve I am the original, first and sole inventor (if oniy one name is listed below) or an original, 
first and joint inventor (if plural names are listed below) of the subject matter which is claimed 
and for which a patent is sought on the invention entitled 
DRIVING-OPERATION ASSIST AND RECORDING MEDIUM 



the specification of which is attached hereto unless the following box is checked: 
was fiied on 06 October 1999 as 

United States Application Number or PCT International Application Number PCT/JP 99/05509 

and was amended on (if applicable). 

I hereby state that I have reviewed and understand the contents of the above identified 
specification, including the claims, as amended by any amendment referred to above. 

J acknowledge the duty to disclose information which is material to patentability as defined in 37 
CFR § 1.56. 

I hereby claim foreign priority benefits under 35 U.S.C. §l19(a)-{d) or § 365(b) of any foreign 
application(s) for patent or inventor's certificate, or § 365(a) of any PCT International application 
which designated at least one country other than the United States, listed below and have also 
identified below by checking the box, any foreign application for patent or inventor's certificate, 
or PCT International application having a filing date before that of the application on which priority 
is claimed: 

Prior Foreign Appiication(s) Priority Not Claimed 

H 10-2 86, 246 JAPAN Octob er 8, 1998 

(Number) (Country) (Day/Month/Year Filed) | | 

H 10-3 49, 107 JAPAN Decem ber 8, 1998 

(Number) (Country) (Day/Month/Year Filed) 

hereby claim the benefit under 35 U.S.C. § 1 19(e) of any United States provisional appiication(s) 
listed beiow. 

{Application Number) (Filing Date) 

(Application Number) (Filing Date) 

I hereby claim the benefit under 35 U.S.C. § 120 of any United States application(s), or 365(c) of 
any PCT International application designating the United States, listed below and, insofar as the 
subject matter of each of the claims of this application is not disclosed in the prior United States 
or PCT International application in the manner provided by the first paragraph of 35 U.S.C. § 1 12, 
I acknowledge the duty to disclose information which is material to patentability as defined in 37 
CFR § 1.56 which became available between the filing date of the prior application and the 
national or PCT international filing date of this application: 



Form PTO/SB/01 (6-95)[R&P) 



Patent and Trademark Office-U.S. DEPARTMENT OF COMMERCE 



ft 



(Application Number) (Filing Date) (Status - patented, pending, abandoned) 



(Application Number) (Filing Date) (Status - patented, pending, abandoned) 

POWER OF ATTORNEY: As a named inventor, I hereby appoint the following attorney (s) and/or 
agent(s) to prosecute this application and transact ail business in the Patent and Trademark Office 
connected therewith: 

Paul F. Prestia Reg.No,. 23.031 Lawrence E. Ashery Reg.N o. 34,515 Mark J. Marcelli 

Allan Ratner Reg.No -J&Z12L. Robert L Anderson Reg. No . 25,771 Christopher J. Dervish ian 

Andrew L Ney Reg.N o. 20,300 Christopher R. Lewis Reg.N o. 36.201 

Kenneth N. Nigon Reg.Nq,3L£4j9_ Louis W. Beardell, Jr. Reg. No. 40,506 _ 

Kevin R. Casey Reg.NqJ22JJ7_ Rocco L. Adomato Reg.N o. 40,480 

Benjamin E. Leace Reg.N o. 33.412 Jacques L. Etkowicz 

James C. Simmons Reg.No. 24J342 Eric A. Dichter 

Address all correspondence to: 

3 .Ratner & P restia. Suit e 301 , One WWIakPS Bervwn, P £L_Bj3xi3fln Valley Fn rgg PA iQ4a2=03aQ_ 
Address all telephone calls to: at (610) 407-0700. 

4 hereby declare that ail statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willfui false statements and the like so 
= made are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the 
"United States Code and that such willfui false statements may jeopardize the validity of the 
=application or any patent issued thereon. 



full name of sole or first inventor (given name, family name) Shusaku 0KAM0T 0 
inventor's signature s 

Date May 30, 2 000 



Residence Hira kata-shi, Osaka JAPAN ~T~£H 
Citizenship JAPA N " 

Post Office Address 3-6- 1 Q-2 15, Hoshigaoka, Hirakata-shi, Osaka 573-0013 JAPAN 



Full name of second joint inventor, if any (given name, family name) Hirof umi ISHII 

Second Inventor's signature ; Ivw ^ ^ Date May 30, 2000 

Residence Nar a~shi, Nara JAI^AN 
Citizenship JAPA N ° ' 

Post Office Address 226- 5, Nishikitsu ji-cho, Nara-shi, Nara 631-8325 JAPAN 



Additional inventors are being named on separately numbered sheets attached hereto. 
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Ftil! name of third joint inventor, if any (given name, family name) Masamj j: 

Third inventor's signature ^_ Dal 
,Q Residence Hira kata-shi, Osaka JAPAN ~xP% 
0 Citizenship JAPAN ~ ~~ J f A 

Post Office Address 22-5 -304, Fu jisaka-kitamachi, Hirakata-shi, 
Osak a 573-0151 JAPAN 

Full name of fourth joint inventor, if any {given name, family name) Kunio NOBORI 

Fourth inventor's signature y — 

Residence 

ado ma-shi, Osaka JAPAN —rCH 
Citizenship JAPAN k 
Post Office Address 16-1 -811, Jyousyou ji~cho, Kadoma-shi, Osaka 571-0063 JAPAN 



full name of fifth joint inventor, if any (given name, family name) A t s U s jii_JiQRIM JR A 
; fifth inventor's signature M^u*~w^ . Date Ma y 30 < 2000 

Residence Nara -shi, NARA J APAN 
; Citizenship JAPAN 

j^ost Office Address 14-8 , Nishitomigaoka 4-chome, Nara-shi, NARA 631-0006 JAPAN 



full name of sixth joint inventor, if any (given name, family name) 



Ipixth inventor's signature _ 

Residence 

Citizenship 

Post Office Address 



Full name of seventh joint inventor, if any (given name, family name) 

Seventh inventor's signature Date 

Residence 

Citizenship 

Post Office Address 
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